Question 1 (Show main steps of your work to get full points)

The \((X'X)^{-1}\) for the \(y=β_0+β_1 x_1+β_2 x_2+β_3 x_3+β_4 x_4+β_5 x_5+β_6 x_6+ε\) is given below.

  1. If MSE = 1.395 and n = 38 , compute the (Keep 4 or more decimal places, DO NOT round in the intermediate steps)

  2. se(β ̂_4)

\[se(\mathbf{\hat\beta_4})=\sqrt{MSE\times C_{55}}=\sqrt{1.395\times0.069}=0.3102499\]

  1. Cov(β ̂_2,β ̂_4)

\[Cov(\mathbf{\hat\beta_2,\hat\beta_4})=MSE\times C_{35}=1.395\times(-0.035)=-0.048825\]

  1. Cor(β ̂_2,β ̂_4 )

\[se(\mathbf{\hat\beta_2})=\sqrt{MSE\times C_{33}}=\sqrt{1.395\times0.067}=0.3057205\]

\[Cor(\mathbf{\hat\beta_2,\hat\beta_4})=\frac{Cov(\mathbf{\hat\beta_2,\hat\beta_4})}{se(\mathbf{\hat\beta_2})se(\mathbf{\hat\beta_4})}=\frac{-0.048825}{0.3057205\times0.3102499}=-0.5147615\]

  1. Without computing anything, explain which estimator is the most consistent.

\(C_{66}=0.058\) has the smallest value. \(\hatβ_5\) has the the least variance and the most consistent among the estimators.

  1. Without computing anything , list the pair(s) of estimators that are positively correlated. Provide a reason.

According to the \((X'X)^{(-1)}\),

\(C_{13},\ C_{17},\ C_{24},\ C_{25},\ C_{67}\) are positive.

The positively correlated pairs of parameters are

\(\hatβ_0\) and \(\hatβ_2\), \(\hatβ_0\) and \(\hatβ_6\), \(\hatβ_1\) and \(\hatβ_3\), \(\hatβ_1\) and \(\hatβ_4\), \(\hatβ_5\) and \(\hatβ_6\).

  1. Consider the following hypothesis: \(H_0: β_1=2β_3,β_2=β_3,β_5=0\)

  2. Report the T matrix, β vector and c vector along with their dimensions, and the rank of T matrix for testing the above hypothesis.

\[ \mathbf{T}=\begin{bmatrix} 0 & 1 & 0 & -2 & 0 & 0& 0 \\ 0 & 0 & 1 & -1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 1 & 0 \end{bmatrix}_{3\times7} \mathbf{β}=\begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \\ \beta_3 \\ \beta_4 \\ \beta_5 \\ \beta_6 \end{bmatrix}_{7\times1} \mathbf{C}=\begin{bmatrix} 0 \\ 0 \\ 0\end{bmatrix}_{3\times1} rank(T)=3 \]

  1. Report the values of numerator and denominator degrees of freedom for the corresponding F test. For F test, the numerator is MSR while denominator is MSE, thus

In this hypothesis,\(y=β_0+2β_3x_1+β_3x_2+β_3x_3+β_4x_4+0x_5+β_6x_6+ε=β_0+β_3(2x_1+x_2+x_3)+β_4x_4+β_6x_6+ε\)

The value of numerator is \(r=df_{Reduced}-df_{Full}=n-(3+1)-[n-(6+1)]=3\)

The denominator degrees of freedom is \(df_{Full}=n-(k+1)=38-(6+1)=31\)

  1. Show the following equation is an alternative form of the sum of squares of regreesion or model (SSR).

\[SSR=\sum_{i=1}^n(\hat y_i-\bar y)^2=\sum_{i=1}^n(\hat y_i^2-2\hat y_i\bar y+\bar y^2)=\sum_{i=1}^n\hat y_i^2-2\bar y\sum_{i=1}^n\hat y_i+\sum_{i=1}^n\bar y^2\]

\[=\sum_{i=1}^n\hat y_i^2-2\bar yn\frac{\sum_{i=1}^n\hat y_i}n+n\bar y^2=\sum_{i=1}^n\hat y_i^2-2\bar yn\bar y+n\bar y^2=\sum_{i=1}^n\hat y_i^2-n\bar y^2\]

Question 2 (Use software to analyze the given data)

The data in the WaterFlow file are simulated data on peak rate of flow (in cfs) of water from six watersheds following storm episodes. The predictors are:

x1 : Area of watershed (mi2) x2 : Area impervious to water (mi2)
x3 : Average slope of watershed (percent)
x4 : Longest stream flow in watershed (1000s of feet)
x5 : surface absorbency index, (0= complete absorbency, 100=no absorbency)
x6 : estimated soil storage capacity (inches of water)
x7 : Infiltration rate of water into soil (inches/hour)
x8 : Rainfall (inches)
x9 : Time period during which rainfall exceeded ¼ inch/hour

  1. Create the matrix of scatterplots and compute the correlation matrix for all the variables. Copy and paste them here.


  1. Based on scatterplots and correlation, explain which predictors are significantly related to (most likely to contribute to the variation in) the response variable.

Based on scatterplots and correlation, X2(0.666),X7(0.668),X1(0.781),X4(0.866) have medium to strong positive linear relationship to the response variable (Correlation coefficient is more than 0.6). X5(-0.62) have medium negative linear relationship to the response variable.


  1. Fit the full model.

\[\hat y=292.561-203.144X_1+ 1055.782X_2-49.24X_3+209.762X_4-10.197X_5-24.558X_6+142.778X_7+511.713X_8-301.872X_9\]


  1. Explain whether the overall model is significant at 5% significance level.

The fitted model is statistically significant at 5% significance level (p-value=0.0000). But most of the coefficients are not significent. This model is not the best fitted model.


  1. Explain whether assumptions of random errors and model are satisfied. If there is a violation of those, then suggest reasonable methods to correct them.
  • Residual Diagnostics: Use plots to examine residuals to validate OLS assumptions

There is some violation of assumptions about the errors:

On the residual plot, there is a funnel pattern.

On the outlier and leverage plot, there are two outlier.

On the qq plot, most of points follow approximately straight line but have some positive skew.

  • Suggestion: Transform and other diagnostics.

I suggest using natural log of response to make a variance-stabilizing transformations.

Other diagnostics of heteroskedasticity, variable selection, measures of influence also should be considered.


  1. How much of the sum of squares is explained by rainfall, given that all the other regression coefficients are in the model?

Accroding to the F test, the partial sum of squares explained by rainfall is 2209825, given that all the other regression coefficients are in the model.


  1. Explain whether there is a problem of multicollinearity.
  • Collinearity diagnostics:

The model does have serious problems of multicollinearity. The VIF of variables X4(105.754708), X1(101.859709), X3(31.446394 ), X7(20.53505) are larger than 10.

It will be important to solve multicollinearity. However, X7, X1, and X4 have medium to strong positive linear relationship to the response variable. It is also dangerous to remove these variables. We should have more diagnostics and comparisons.


  1. Interpret the estimated coefficient of rainfall predictor of the full model using question context.

Coefficient of 511.713 suggests the peak rate of flow increases by 511.713 cubic feet per second when the rainfall increases by 1 inch and other variables are constants.


  1. Create a new variable using natural log of response. Then fit the full model using this new variable as response.

\[\hat y=3.402256-0.013532X_1-1.023664X_2+0.177966X_3+0.108788X_4-0.009622X_5-0.389474X_6+4.233475X_7+0.63007X_8-0.462276X_9\]

  1. Explain whether the overall model is significant at 5% significance level.

The fitted model is statistically significant at 5% significance level (p-value=0000). But most of the coefficients are not significent. This model is not the best fitted model.


  1. Explain whether there is a problem of multicollinearity.

The variance-stabilizing transformations does not change the problem of multicollinearit.

The model still has serious problems of multicollinearity. The VIF of variables X4(105.754708), X1(101.859709), X3(31.446394), X7(20.53505) are larger than 10.

It will be important to solve multicollinearity. However, X7, X1, and X4 have medium to strong positive linear relationship to the response variable. It is also dangerous to remove these variables. We should have more diagnostics and comparisons.


  1. If you wanted to simplify this full model, explain which predictor you would eliminate first.

If just considering the VIF, X4 has the maximum value (105.754708) or X1(101.859709) is the first to remove.

However, according to the correlation coefficients, both X4(0.866) and X1(0.781) is strongly correlated with y. In this way, the third multicollinear X3(31.446394) with a weak relationship with y (0.205) should be removed.

According to the variable names of X4, X1, and X3, they are geographic variables. Predictor X1 is the area of watershed while X4 is the longest stream flow in watershed, x3 is the average slope of watershed. For the given 6 watersheds, X1 and X4 are strongly related. A high correlation (0.921) is expected between these two variables. But X3 is not significently related with X1(-0.078) or X4(0.245). Removing X3 will make more sense.

Actrually, I don’t agree remove any predictor in this stage. Since removing any predictor can draw down the VIF significently. in this manner, the predictor weakest related with y, X6(0.0453) shoudle be first considered.

The textbook suggest that the general approaches for dealing with multicollinearity include collecting additional data, model respecification (redefine the regressors, variable elimination), estimation methods (Ridge Regression, Principal-Component Regression). “Variable elimination is often a highly effective technique. However, it may not provide a satisfactory solution if the regressors dropped from the model have significant explanatory power relative to the response y. That is, eliminating regressors to reduce multicollinearity may damage the predictive power of the model.” (Montgomery et al., 2012. p.304)

In fact, after elimination regression, the multicollinearity dissapeared in all the models. We should gather sufficient evidents before removing any predictor. I will disscuss it at the end.


  1. Use the forward selection method to find the best model (use α=0.15) and report the final fitted model with estimated coefficients here.

Use Stepwise Forward Regression based on p values (use α=0.15)

\[\hat y=2.872+0.168X_3+0.122X_4+3.106X_7\]

Use Stepwise AIC Forwardd Regression

\[\hat y=2.692+0.184X_3+0.109X_4+-0.368X_6+4.085X_7+0.612X_8+-0.448X_9\]


  1. Use the backward elimination method to find the best model (use α=0.05) and report the final fitted model with estimated coefficients here.

Stepwise Backward Regression based on p values (use α=0.05) and Stepwise AIC Backward Regression have same results.

\[\hat y=2.692+0.184X_3+0.109X_4+-0.368X_6+4.085X_7+0.612X_8+-0.448X_9\]


  1. Use best subsets method (6 models from each size) to find the best model for these data and report the final fitted model with estimated coefficients here.

Best subsets method gives a same model.

\[\hat y=2.692+0.184X_3+0.109X_4+-0.368X_6+4.085X_7+0.612X_8+-0.448X_9\]


  1. If the final models in the previous 3 methods are different, compare their model adequacy and suggest one best model.
Method By Keep Remove
Stepwise Forward P=0.15 X3, X4, X7 X1,X2,X5,X6,X8,X9
Stepwise Forward AIC X3,X4,X6,X7,X8,X9 X1,X2,X5
Stepwise Backward P=0.05 X3,X4,X6,X7,X8,X9 X1,X2,X5
Stepwise Backward AIC X3,X4,X6,X7,X8,X9 X1,X2,X5
Stepwise Both P X3, X4, X7 X1,X2,X5,X6,X8,X9
Stepwise Both AIC X3,X4,X6,X7,X8,X9 X1,X2,X5
Best Subset / X3,X4,X6,X7,X8,X9 X1,X2,X5
all possible / X3,X4,X6,X7,X8,X9 X1,X2,X5
Model VIF F P-value(F) MSR MSE \(R_{adjusted}^2\) \(R_{Predict}^2\) P-value(t) Residuals Plots
3-4-7 <10 70.378 0.0000 21.188 0.301 0.878 0.854 Max=0.054 Good enough
3-4-6-7-8-9 <10 68.16 0.0000 11.265 0.165 0.933 0.908 Max=0.019 Good enough

Both models do not have a problem of multicollinearity (VIF <10), and small P-values for F test. Compared to the model with only x3, x4 and x7, the model with 6 predictors has a slightly higher (about by 6%) adjusted R square and higher (about by 5%) prediction R-square, which means it shows stronger predictive capability.

However, all the 6 predictors in a model are statistically significant at 2% significance level (the maximum p-values are 0.019, respectively). In the 3-variable model, X7 get a high p-value (0.054) which means not significant at 5% significance level. If we change the p-value as the parameter of forward selection, the same model will happened for 0.5<a<0.16.

Further, violation of assumptions about the errors (no pattern on residual plots and points follow approximately straight line on the qq plot).

Both of Correlation between observed residuals and expected residuals under normality.The 6-predictor model got 0.9837263 while the 6-predictor model got 0.9856766.

There is no significant pattern on the plot of studentized residuals versus predicted values from the model with only one predictor. The partial regression plots do not show nonlinear patterns and hence first-order terms are good enough.

Finally, the model with 6 predictors has higer predicotion R-square, higher significance level for coefficients. Therefore, the best model will be the model with 6 predictors.


  1. Provide complete ANOVA table for the best model. Provide partial sum of squares, estimated coefficients, standard errors, p-values, 95% Bonferroni joint confidence intervals for the coefficients of the best model. Provide in a tabular form clearly.
~ Model Summary
R 0.973 RMSE (Root Mean Square Error) 0.407
R-Squared 0.947 Coef. Var 6.385
Adj. R-Squared 0.933 MSE (Mean Square Error) 0.165
Pred R-Squared 0.908 MAE (Mean Absolute Error) 0.273
~ ANOVA
Sum of Squares DF Mean Square F Sig.
Regression 67.591 6 11.265 68.16 0.0000
Residual 3.801 23 0.165
Total 71.393 29
~ Parameter Estimates
model Beta Std. Error Std. Beta t Sig lower upper
(Intercept) 2.692 0.445 6.046 0.000 1.771 3.613
X4 0.109 0.026 0.499 4.244 0.000 0.056 0.162
X3 0.184 0.032 0.476 5.698 0.000 0.117 0.251
X7 4.085 1.213 0.406 3.367 0.003 1.575 6.595
X8 0.612 0.133 0.493 4.614 0.000 0.337 0.886
X9 -0.448 0.108 -0.450 -4.135 0.000 -0.672 -0.224
X6 -0.368 0.146 -0.133 -2.526 0.019 -0.669 -0.066

  1. How much variation in the response is explained by the best model after taking number of data and regression coefficients in to account?

67.591


  1. Report the PRESS statistic of the best model.

The value of PRESS is 6.538275. About 90.8% of variation in predicting the peak rate of flow (in cfs) of water from six watersheds following storm episodes.


  1. Report the complete code along with output here.

Singh (1972) used linear models with a logarithm transformation of the variables. We retained the following where the dependent variables can either be total storm flow volume (Qt) in mm, quick flow volume (Qf) in mm or peak flow (Qpk) in m3 sec−1 km−2. Independent variables were storm rainfall (P) in mm, initial flow (Qi) in mm h−1, rainfall frequency (Fp), the inverse of rainfall duration, in h−1 and a dummy variable (CC) representing the treatment effect on basin 7A. CC was 0 and 1 for the calibration (1967–1992) and treated (1994–1998) periods, respectively. β0 to β4 are regression coefficients of the independent variables. All interactions between the independent variables were also tested for significance at α=0.10.

\[ln(Dependent\ Variable)=β_0+β_1lnP+β_2lnQ+β_3lnFP+β_4CC+Interactions+\varepsilon\]

The significance of the regression coefficients (being different from 0) in the models has been tested with a t-test procedure at α=0.10 using the GLM procedure of the SAS system for Windows (SAS Institute, Inc., 1989). A regression coefficient significantly different from 0 for the variable CC indicates that the treatment had a significant effect on the dependent variable. Normality of residuals has been tested using the Shapiro–Wilk test. Selection criteria for all events were the same as for the paired basins approach. However, the rainfall events following night-frost, which may caused localized surface runoff on ice as observed by Prevost et al. (1990), were omitted since these events are too rare to be well represented during the calibration and post-treatment periods. Hence, all events within three weeks following the end of the snowmelt period were not retained. The end of the snowmelt period was obtained from observations in a standard forest snow line at Montmorency Forest. (Guillemette et al., 2005)

  • Residual Diagnostics:

Includes plots to examine residuals to validate OLS assumptions

There is no violation of assumptions about the errors (no pattern on residual plots and points follow approximately straight line on the qq plot).

  • Variable selection:

Differnt variable selection procedures such as all possible regression, best subset regression, stepwise regression, stepwise forward regression and stepwise backward regression

  • Heteroskedasticity:

Tests for heteroskedasticity include bartlett test, breusch pagan test, score test and f test

  • Measures of influence:

Use different plots to detect and identify influential observations

  • Collinearity diagnostics:

VIF, Tolerance and condition indices to detect collinearity and plots for assessing mode fit and contributions of variables

x1 : Area of watershed (mi2)
x8 : Rainfall (inches)

x9 : Time period during which rainfall exceeded ¼ inch/hour
x4 : Longest stream flow in watershed (1000s of feet)

x3 : Average slope of watershed (percent)

x2 : Area impervious to water (mi2)
x5 : surface absorbency index, (0= complete absorbency, 100=no absorbency)
x7 : Infiltration rate of water into soil (inches/hour)
x6 : estimated soil storage capacity (inches of water)

Full model

eliminated model

  • Residual Diagnostics:

Includes plots to examine residuals to validate OLS assumptions

There is no violation of assumptions about the errors (no pattern on residual plots and points follow approximately straight line on the qq plot).

Residual QQ Plot Residual Normality Test Residual vs Fitted Values Plot Residual Histogram

  • Variable selection:

Differnt variable selection procedures such as all possible regression, best subset regression, stepwise regression, stepwise forward regression and stepwise backward regression

  • Heteroskedasticity:

Tests for heteroskedasticity include bartlett test, breusch pagan test, score test and f test

Bartlett Test Breusch Pagan Test Score Test F Test

  • Measures of influence:

Use different plots to detect and identify influential observations

Cook’s D Bar Plot Cook’s D Chart DFBETAs Panel DFFITs Plot Studentized Residual Plot Standardized Residual Chart Studentized Residuals vs Leverage Plot Deleted Studentized Residual vs Fitted Values Plot Hadi Plot Potential Residual Plot

[1]: Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis (Vol. 821). John Wiley & Sons.

[2]: Guillemette, F., Plamondon, A. P., Prévost, M., & Lévesque, D. (2005). Rainfall generated stormflow response to clearcutting a boreal forest: peak flow comparison with 50 world-wide basin studies. Journal of hydrology, 302(1-4), 137-153.

(a) The matrix of scatterplots and the correlation matrix

library(tidyverse)
table_wf <- read_table2("WaterFlow.txt")
library(GGally)
ggpairs(data=table_wf[c(1:10)])

(c) The fitted full model

# build the model
model_wf_full <- lm(y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_regress(model_wf_full)
##                           Model Summary                            
## ------------------------------------------------------------------
## R                       0.906       RMSE                  609.308 
## R-Squared               0.821       Coef. Var              47.188 
## Adj. R-Squared          0.741       MSE                371256.369 
## Pred R-Squared          0.618       MAE                   366.548 
## ------------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                  ANOVA                                   
## ------------------------------------------------------------------------
##                     Sum of                                              
##                    Squares        DF    Mean Square      F         Sig. 
## ------------------------------------------------------------------------
## Regression    34143007.990         9    3793667.554    10.218    0.0000 
## Residual       7425127.376        20     371256.369                     
## Total         41568135.367        29                                    
## ------------------------------------------------------------------------
## 
##                                        Parameter Estimates                                        
## -------------------------------------------------------------------------------------------------
##       model        Beta    Std. Error    Std. Beta      t        Sig          lower        upper 
## -------------------------------------------------------------------------------------------------
## (Intercept)     292.561      4428.618                  0.066    0.948     -8945.373     9530.495 
##          X1    -203.144       410.268       -0.472    -0.495    0.626     -1058.947      652.660 
##          X2    1055.782      9833.700        0.028     0.107    0.916    -19456.957    21568.521 
##          X3     -49.240       156.200       -0.167    -0.315    0.756      -375.067      276.588 
##          X4     209.762       162.046        1.258     1.294    0.210      -128.259      547.783 
##          X5     -10.197        51.088       -0.059    -0.200    0.844      -116.764       96.370 
##          X6     -24.558       303.529       -0.012    -0.081    0.936      -657.709      608.592 
##          X7     142.778      3288.443        0.019     0.043    0.966     -6716.793     7002.349 
##          X8     511.713       209.741        0.541     2.440    0.024        74.200      949.226 
##          X9    -301.872       171.996       -0.398    -1.755    0.095      -660.649       56.905 
## -------------------------------------------------------------------------------------------------
model_wf_full%>% summary()
## 
## Call:
## lm(formula = y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9, 
##     data = table_wf)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1404.21  -318.77    74.73   266.66  1274.30 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)   292.56    4428.62   0.066   0.9480  
## X1           -203.14     410.27  -0.495   0.6259  
## X2           1055.78    9833.70   0.107   0.9156  
## X3            -49.24     156.20  -0.315   0.7558  
## X4            209.76     162.05   1.294   0.2103  
## X5            -10.20      51.09  -0.200   0.8438  
## X6            -24.56     303.53  -0.081   0.9363  
## X7            142.78    3288.44   0.043   0.9658  
## X8            511.71     209.74   2.440   0.0241 *
## X9           -301.87     172.00  -1.755   0.0945 .
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 609.3 on 20 degrees of freedom
## Multiple R-squared:  0.8214, Adjusted R-squared:  0.741 
## F-statistic: 10.22 on 9 and 20 DF,  p-value: 9.744e-06
Anova(model_wf_full)
## Anova Table (Type II tests)
## 
## Response: y
##            Sum Sq Df F value  Pr(>F)  
## X1          91022  1  0.2452 0.62589  
## X2           4279  1  0.0115 0.91557  
## X3          36893  1  0.0994 0.75585  
## X4         622091  1  1.6756 0.21025  
## X5          14790  1  0.0398 0.84381  
## X6           2430  1  0.0065 0.93632  
## X7            700  1  0.0019 0.96580  
## X8        2209825  1  5.9523 0.02414 *
## X9        1143622  1  3.0804 0.09455 .
## Residuals 7425127 20                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(c) ii Residual diagnostics

#Model Fit Assessment
ols_plot_diagnostics(model_wf_full)

# Part & Partial Correlations
ols_test_correlation(model_wf_full) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9710713
# Residual Normality Test
ols_test_normality(model_wf_full) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9589         0.2898 
## Kolmogorov-Smirnov        0.1423         0.5314 
## Cramer-von Mises          2.5333         0.0000 
## Anderson-Darling          0.5169         0.1748 
## -----------------------------------------------

(c) iii The partial regression and nonlinear diagnostics

#Lack of Fit F Test
ols_pure_error_anova(lm(y~X8, data = table_wf))
## Lack of Fit F Test 
## ---------------
## Response :   y 
## Predictor:   X8 
## 
##                        Analysis of Variance Table                         
## -------------------------------------------------------------------------
##                 DF      Sum Sq        Mean Sq      F Value       Pr(>F)   
## -------------------------------------------------------------------------
## X8               1     4616882.92    4616882.92    5.795558    0.02290414 
## Residual        28    36951252.44    1319687.59                           
##  Lack of fit    21    31374881.28    1494041.97    1.875466     0.2003839 
##  Pure Error      7     5576371.17     796624.45                           
## -------------------------------------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_full)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_full)

(c) iv Collinearity diagnostics

# for full model
ols_coll_diag(model_wf_full)
## Tolerance and Variance Inflation Factor
## ---------------------------------------
## # A tibble: 9 x 3
##   Variables Tolerance    VIF
##   <chr>         <dbl>  <dbl>
## 1 X1          0.00982 102.  
## 2 X2          0.133     7.52
## 3 X3          0.0318   31.4 
## 4 X4          0.00946 106.  
## 5 X5          0.103     9.68
## 6 X6          0.433     2.31
## 7 X7          0.0487   20.5 
## 8 X8          0.182     5.50
## 9 X9          0.174     5.75
## 
## 
## Eigenvalue and Condition Index
## ------------------------------
##      Eigenvalue Condition Index    intercept           X1           X2           X3           X4           X5           X6           X7           X8           X9
## 1  8.3083720047        1.000000 8.462607e-06 5.328594e-05 6.355151e-04 0.0000760217 4.150013e-05 1.417617e-05 0.0008109678 0.0001066112 0.0003687855 0.0004151973
## 2  0.9146513033        3.013909 3.727606e-05 1.954057e-03 1.170272e-02 0.0005739095 4.651577e-04 1.081786e-04 0.0035456399 0.0001817229 0.0014730291 0.0011848948
## 3  0.3157077198        5.129976 6.956546e-06 1.601785e-05 8.257711e-05 0.0097446616 4.781482e-04 6.076062e-09 0.0081810709 0.0002927965 0.0175774490 0.0291828488
## 4  0.2027798531        6.400967 1.163563e-04 3.691673e-04 2.265111e-02 0.0069919127 2.025445e-03 4.207282e-04 0.0594054477 0.0037315932 0.0110815180 0.0196917886
## 5  0.1283540243        8.045503 1.755786e-04 5.211575e-03 9.236831e-02 0.0009818635 6.328213e-05 7.604150e-04 0.2692817619 0.0006913700 0.0007246206 0.0003261699
## 6  0.0839205416        9.950017 8.653324e-04 5.816786e-03 2.768292e-01 0.0026483249 1.709813e-03 1.680493e-03 0.0721703877 0.0040196108 0.0166605115 0.0059859638
## 7  0.0244574411       18.431151 1.228356e-03 3.944752e-02 5.573983e-02 0.0007406723 7.931546e-03 3.329908e-03 0.0049097386 0.1485064761 0.0995363708 0.0995227734
## 8  0.0172641365       21.937419 6.575462e-04 7.426534e-03 4.085809e-02 0.0003554064 4.944025e-03 2.364272e-04 0.0098681982 0.0384834510 0.7571879031 0.7489277292
## 9  0.0041546428       44.718901 7.061743e-03 1.591436e-01 4.939182e-02 0.1757973907 3.445438e-01 3.963894e-02 0.3794245842 0.2469894804 0.0193633680 0.0055230478
## 10 0.0003383328      156.706094 9.898424e-01 7.805615e-01 4.497409e-01 0.8020898367 6.377973e-01 9.538107e-01 0.1924022031 0.5569968879 0.0760264445 0.0892395862

(d) The fitted log model

# build full log model
model_wf_full_log <- lm(log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_regress(model_wf_full_log)
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.973       RMSE               0.433 
## R-Squared               0.947       Coef. Var          6.808 
## Adj. R-Squared          0.924       MSE                0.188 
## Pred R-Squared          0.886       MAE                0.265 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     67.635         9          7.515    40.002    0.0000 
## Residual        3.757        20          0.188                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                    Parameter Estimates                                    
## -----------------------------------------------------------------------------------------
##       model      Beta    Std. Error    Std. Beta      t        Sig       lower     upper 
## -----------------------------------------------------------------------------------------
## (Intercept)     3.402         3.150                  1.080    0.293     -3.169     9.974 
##          X1    -0.014         0.292       -0.024    -0.046    0.963     -0.622     0.595 
##          X2    -1.024         6.995       -0.021    -0.146    0.885    -15.615    13.568 
##          X3     0.178         0.111        0.461     1.602    0.125     -0.054     0.410 
##          X4     0.109         0.115        0.498     0.944    0.357     -0.132     0.349 
##          X5    -0.010         0.036       -0.042    -0.265    0.794     -0.085     0.066 
##          X6    -0.389         0.216       -0.141    -1.804    0.086     -0.840     0.061 
##          X7     4.233         2.339        0.421     1.810    0.085     -0.646     9.113 
##          X8     0.630         0.149        0.508     4.223    0.000      0.319     0.941 
##          X9    -0.462         0.122       -0.465    -3.778    0.001     -0.717    -0.207 
## -----------------------------------------------------------------------------------------
#Model Fit Assessment
ols_plot_diagnostics(model_wf_full_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_full_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9808603
# Residual Normality Test
ols_test_normality(model_wf_full_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9659         0.4341 
## Kolmogorov-Smirnov        0.0993         0.9007 
## Cramer-von Mises          4.948          0.0000 
## Anderson-Darling          0.3686         0.4062 
## -----------------------------------------------

(d) (2) Collinearity diagnostics

library(MASS)
# for log model
ols_vif_tol(model_wf_full_log)
## # A tibble: 9 x 3
##   Variables Tolerance    VIF
##   <chr>         <dbl>  <dbl>
## 1 X1          0.00982 102.  
## 2 X2          0.133     7.52
## 3 X3          0.0318   31.4 
## 4 X4          0.00946 106.  
## 5 X5          0.103     9.68
## 6 X6          0.433     2.31
## 7 X7          0.0487   20.5 
## 8 X8          0.182     5.50
## 9 X9          0.174     5.75
model_wf_aic_log <- stepAIC(model_wf_full_log)
## Start:  AIC=-42.32
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X1    1    0.0004 3.7577 -44.322
## - X2    1    0.0040 3.7613 -44.293
## - X5    1    0.0132 3.7705 -44.220
## - X4    1    0.1673 3.9246 -43.018
## <none>              3.7573 -42.325
## - X3    1    0.4819 4.2392 -40.705
## - X6    1    0.6113 4.3686 -39.803
## - X7    1    0.6153 4.3726 -39.775
## - X9    1    2.6819 6.4392 -28.164
## - X8    1    3.3503 7.1076 -25.201
## 
## Step:  AIC=-44.32
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X2    1    0.0110 3.7686 -46.234
## - X5    1    0.0267 3.7844 -46.110
## <none>              3.7577 -44.322
## - X6    1    1.0447 4.8023 -38.963
## - X7    1    1.5520 5.3097 -35.950
## - X4    1    1.8469 5.6046 -34.328
## - X9    1    2.8341 6.5918 -29.461
## - X8    1    3.4848 7.2425 -26.637
## - X3    1    5.0955 8.8532 -20.613
## 
## Step:  AIC=-46.23
## log(y) ~ X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X5    1    0.0327 3.8013 -47.975
## <none>              3.7686 -46.234
## - X6    1    1.0375 4.8061 -40.939
## - X4    1    1.8741 5.6428 -36.125
## - X7    1    1.9036 5.6722 -35.968
## - X9    1    2.8353 6.6040 -31.406
## - X8    1    3.4744 7.2430 -28.635
## - X3    1    5.1264 8.8951 -22.471
## 
## Step:  AIC=-47.98
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## <none>              3.8013 -47.975
## - X6    1    1.0542 4.8555 -42.632
## - X7    1    1.8739 5.6752 -37.953
## - X9    1    2.8256 6.6270 -33.302
## - X4    1    2.9771 6.7784 -32.624
## - X8    1    3.5182 7.3195 -30.320
## - X3    1    5.3653 9.1666 -23.569
ols_vif_tol(model_wf_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X3            0.332  3.01
## 2 X4            0.167  5.97
## 3 X6            0.839  1.19
## 4 X7            0.159  6.28
## 5 X8            0.202  4.94
## 6 X9            0.195  5.12
# remove X4
model_wf_rm4_log <- lm(log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm4_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.119  8.38
## 2 X2            0.209  4.79
## 3 X3            0.379  2.64
## 4 X5            0.187  5.35
## 5 X6            0.836  1.20
## 6 X7            0.165  6.05
## 7 X8            0.187  5.35
## 8 X9            0.183  5.45
model_wf_rm4_aic_log <- stepAIC(model_wf_rm4_log)
## Start:  AIC=-43.02
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X5    1    0.0457  3.9703 -44.671
## - X2    1    0.1508  4.0754 -43.887
## <none>               3.9246 -43.018
## - X1    1    1.6800  5.6046 -34.328
## - X6    1    2.1914  6.1160 -31.708
## - X9    1    2.5158  6.4404 -30.158
## - X8    1    3.1937  7.1183 -27.156
## - X7    1    4.3217  8.2463 -22.743
## - X3    1   14.0627 17.9873   0.654
## 
## Step:  AIC=-44.67
## log(y) ~ X1 + X2 + X3 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X2    1    0.1126  4.0829 -45.832
## <none>               3.9703 -44.671
## - X6    1    2.5195  6.4898 -31.929
## - X9    1    2.7581  6.7284 -30.846
## - X1    1    2.7838  6.7541 -30.731
## - X8    1    3.6308  7.6011 -27.187
## - X7    1    4.2769  8.2472 -24.740
## - X3    1   24.3256 28.2959  12.246
## 
## Step:  AIC=-45.83
## log(y) ~ X1 + X3 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## <none>               4.0829 -45.832
## - X6    1    2.4147  6.4976 -33.893
## - X9    1    2.6501  6.7330 -32.825
## - X1    1    2.6955  6.7784 -32.624
## - X8    1    3.5347  7.6176 -29.122
## - X7    1    5.2580  9.3409 -23.004
## - X3    1   25.3225 29.4054  11.399
ols_vif_tol(model_wf_rm4_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.279  3.58
## 2 X3            0.768  1.30
## 3 X6            0.917  1.09
## 4 X7            0.251  3.99
## 5 X8            0.202  4.94
## 6 X9            0.196  5.10
# remove X1
model_wf_rm1_log <- lm(log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm1_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X2            0.245  4.08
## 2 X3            0.318  3.14
## 3 X4            0.115  8.70
## 4 X5            0.283  3.54
## 5 X6            0.717  1.39
## 6 X7            0.118  8.46
## 7 X8            0.190  5.27
## 8 X9            0.185  5.41
model_wf_rm1_aic_log <- stepAIC(model_wf_rm1_log)
## Start:  AIC=-44.32
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X2    1    0.0110 3.7686 -46.234
## - X5    1    0.0267 3.7844 -46.110
## <none>              3.7577 -44.322
## - X6    1    1.0447 4.8023 -38.963
## - X7    1    1.5520 5.3097 -35.950
## - X4    1    1.8469 5.6046 -34.328
## - X9    1    2.8341 6.5918 -29.461
## - X8    1    3.4848 7.2425 -26.637
## - X3    1    5.0955 8.8532 -20.613
## 
## Step:  AIC=-46.23
## log(y) ~ X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X5    1    0.0327 3.8013 -47.975
## <none>              3.7686 -46.234
## - X6    1    1.0375 4.8061 -40.939
## - X4    1    1.8741 5.6428 -36.125
## - X7    1    1.9036 5.6722 -35.968
## - X9    1    2.8353 6.6040 -31.406
## - X8    1    3.4744 7.2430 -28.635
## - X3    1    5.1264 8.8951 -22.471
## 
## Step:  AIC=-47.98
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## <none>              3.8013 -47.975
## - X6    1    1.0542 4.8555 -42.632
## - X7    1    1.8739 5.6752 -37.953
## - X9    1    2.8256 6.6270 -33.302
## - X4    1    2.9771 6.7784 -32.624
## - X8    1    3.5182 7.3195 -30.320
## - X3    1    5.3653 9.1666 -23.569
ols_vif_tol(model_wf_rm1_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X3            0.332  3.01
## 2 X4            0.167  5.97
## 3 X6            0.839  1.19
## 4 X7            0.159  6.28
## 5 X8            0.202  4.94
## 6 X9            0.195  5.12
# remove X3
model_wf_rm3_log <- lm(log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm3_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1           0.0983 10.2 
## 2 X2           0.243   4.11
## 3 X4           0.113   8.87
## 4 X5           0.272   3.68
## 5 X6           0.767   1.30
## 6 X7           0.206   4.85
## 7 X8           0.190   5.26
## 8 X9           0.187   5.36
model_wf_rm3_aic_log <- stepAIC(model_wf_rm3_log)
## Start:  AIC=-40.7
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X7    1    0.1337  4.3729 -41.773
## - X6    1    0.1858  4.4250 -41.418
## <none>               4.2392 -40.705
## - X2    1    0.2995  4.5388 -40.656
## - X5    1    1.1498  5.3891 -35.505
## - X9    1    3.5480  7.7872 -24.461
## - X8    1    4.0900  8.3292 -22.443
## - X1    1    4.6140  8.8532 -20.613
## - X4    1   13.7481 17.9873   0.654
## 
## Step:  AIC=-41.77
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X6    1    0.1369  4.5098 -42.849
## <none>               4.3729 -41.773
## - X2    1    0.6577  5.0306 -39.570
## - X5    1    1.0236  5.3965 -37.464
## - X9    1    3.4161  7.7890 -26.455
## - X8    1    3.9564  8.3293 -24.442
## - X1    1    4.7933  9.1662 -21.570
## - X4    1   13.8200 18.1929  -1.005
## 
## Step:  AIC=-42.85
## log(y) ~ X1 + X2 + X4 + X5 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## <none>               4.5098 -42.849
## - X2    1    0.6110  5.1208 -41.037
## - X5    1    0.8871  5.3969 -39.461
## - X9    1    3.2799  7.7896 -28.452
## - X8    1    3.8347  8.3444 -26.388
## - X1    1    5.0057  9.5155 -22.448
## - X4    1   15.9600 20.4698   0.533
ols_vif_tol(model_wf_rm3_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.119  8.39
## 2 X2            0.313  3.19
## 3 X4            0.123  8.12
## 4 X5            0.343  2.92
## 5 X8            0.209  4.79
## 6 X9            0.205  4.88
# remove X7
model_wf_rm7_log <- lm(log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm7_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1           0.0238 42.0 
## 2 X2           0.310   3.22
## 3 X3           0.135   7.42
## 4 X4           0.0321 31.1 
## 5 X5           0.164   6.09
## 6 X6           0.740   1.35
## 7 X8           0.184   5.45
## 8 X9           0.177   5.66
model_wf_rm7_aic_log <- stepAIC(model_wf_rm7_log)
## Start:  AIC=-39.78
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X3    1    0.0003 4.3729 -41.773
## - X6    1    0.1309 4.5035 -40.891
## <none>              4.3726 -39.775
## - X5    1    0.5584 4.9309 -38.170
## - X2    1    0.6554 5.0280 -37.585
## - X1    1    0.9371 5.3097 -35.950
## - X9    1    3.0659 7.4385 -25.836
## - X8    1    3.6837 8.0563 -23.442
## - X4    1    3.8737 8.2463 -22.743
## 
## Step:  AIC=-41.77
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X6    1    0.1369  4.5098 -42.849
## <none>               4.3729 -41.773
## - X2    1    0.6577  5.0306 -39.570
## - X5    1    1.0236  5.3965 -37.464
## - X9    1    3.4161  7.7890 -26.455
## - X8    1    3.9564  8.3293 -24.442
## - X1    1    4.7933  9.1662 -21.570
## - X4    1   13.8200 18.1929  -1.005
## 
## Step:  AIC=-42.85
## log(y) ~ X1 + X2 + X4 + X5 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## <none>               4.5098 -42.849
## - X2    1    0.6110  5.1208 -41.037
## - X5    1    0.8871  5.3969 -39.461
## - X9    1    3.2799  7.7896 -28.452
## - X8    1    3.8347  8.3444 -26.388
## - X1    1    5.0057  9.5155 -22.448
## - X4    1   15.9600 20.4698   0.533
ols_vif_tol(model_wf_rm7_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.119  8.39
## 2 X2            0.313  3.19
## 3 X4            0.123  8.12
## 4 X5            0.343  2.92
## 5 X8            0.209  4.79
## 6 X9            0.205  4.88
# remove X5
model_wf_rm5_log <- lm(log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm5_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1           0.0269 37.2 
## 2 X2           0.210   4.77
## 3 X3           0.0836 12.0 
## 4 X4           0.0171 58.4 
## 5 X6           0.485   2.06
## 6 X7           0.0774 12.9 
## 7 X8           0.200   5.01
## 8 X9           0.190   5.25
model_wf_rm5_aic_log <- stepAIC(model_wf_rm5_log)
## Start:  AIC=-44.22
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X1    1    0.0139 3.7844 -46.110
## - X2    1    0.0279 3.7983 -45.999
## - X4    1    0.1998 3.9703 -44.671
## <none>              3.7705 -44.220
## - X6    1    0.7524 4.5229 -40.762
## - X7    1    1.1605 4.9309 -38.170
## - X3    1    1.6186 5.3891 -35.505
## - X9    1    2.8181 6.5886 -29.476
## - X8    1    3.5442 7.3147 -26.339
## 
## Step:  AIC=-46.11
## log(y) ~ X2 + X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X2    1    0.0170 3.8013 -47.975
## <none>              3.7844 -46.110
## - X6    1    1.0707 4.8551 -40.635
## - X7    1    1.5504 5.3348 -37.808
## - X9    1    2.8243 6.6087 -31.384
## - X4    1    2.9697 6.7541 -30.731
## - X8    1    3.5305 7.3149 -28.339
## - X3    1    5.3638 9.1482 -21.629
## 
## Step:  AIC=-47.98
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## <none>              3.8013 -47.975
## - X6    1    1.0542 4.8555 -42.632
## - X7    1    1.8739 5.6752 -37.953
## - X9    1    2.8256 6.6270 -33.302
## - X4    1    2.9771 6.7784 -32.624
## - X8    1    3.5182 7.3195 -30.320
## - X3    1    5.3653 9.1666 -23.569
ols_vif_tol(model_wf_rm5_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X3            0.332  3.01
## 2 X4            0.167  5.97
## 3 X6            0.839  1.19
## 4 X7            0.159  6.28
## 5 X8            0.202  4.94
## 6 X9            0.195  5.12
# remove X2
model_wf_rm2_log <- lm(log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9, data=table_wf)
ols_vif_tol(model_wf_rm2_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1           0.0181 55.2 
## 2 X3           0.0583 17.2 
## 3 X4           0.0148 67.4 
## 4 X5           0.163   6.13
## 5 X6           0.543   1.84
## 6 X7           0.114   8.79
## 7 X8           0.183   5.46
## 8 X9           0.175   5.72
model_wf_rm2_aic_log <- stepAIC(model_wf_rm2_log)
## Start:  AIC=-44.29
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X1    1    0.0073 3.7686 -46.234
## - X5    1    0.0370 3.7983 -45.999
## <none>              3.7613 -44.293
## - X4    1    0.3141 4.0754 -43.887
## - X6    1    0.7115 4.4729 -41.095
## - X3    1    0.7775 4.5388 -40.656
## - X7    1    1.2667 5.0280 -37.585
## - X9    1    2.7122 6.4735 -30.005
## - X8    1    3.4001 7.1614 -26.975
## 
## Step:  AIC=-46.23
## log(y) ~ X3 + X4 + X5 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X5    1    0.0327 3.8013 -47.975
## <none>              3.7686 -46.234
## - X6    1    1.0375 4.8061 -40.939
## - X4    1    1.8741 5.6428 -36.125
## - X7    1    1.9036 5.6722 -35.968
## - X9    1    2.8353 6.6040 -31.406
## - X8    1    3.4744 7.2430 -28.635
## - X3    1    5.1264 8.8951 -22.471
## 
## Step:  AIC=-47.98
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9
## 
##        Df Sum of Sq    RSS     AIC
## <none>              3.8013 -47.975
## - X6    1    1.0542 4.8555 -42.632
## - X7    1    1.8739 5.6752 -37.953
## - X9    1    2.8256 6.6270 -33.302
## - X4    1    2.9771 6.7784 -32.624
## - X8    1    3.5182 7.3195 -30.320
## - X3    1    5.3653 9.1666 -23.569
ols_vif_tol(model_wf_rm2_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X3            0.332  3.01
## 2 X4            0.167  5.97
## 3 X6            0.839  1.19
## 4 X7            0.159  6.28
## 5 X8            0.202  4.94
## 6 X9            0.195  5.12
# remove X8
model_wf_rm8_log <- lm(log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9, data=table_wf)
ols_vif_tol(model_wf_rm8_log)
## # A tibble: 8 x 3
##   Variables Tolerance    VIF
##   <chr>         <dbl>  <dbl>
## 1 X1          0.0103   97.5 
## 2 X2          0.134     7.46
## 3 X3          0.0333   30.0 
## 4 X4          0.00973 103.  
## 5 X5          0.114     8.81
## 6 X6          0.435     2.30
## 7 X7          0.0492   20.3 
## 8 X9          0.879     1.14
model_wf_rm8_aic_log <- stepAIC(model_wf_rm8_log)
## Start:  AIC=-25.2
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9
## 
##        Df Sum of Sq    RSS     AIC
## - X9    1   0.00002 7.1076 -27.201
## - X4    1   0.01076 7.1183 -27.156
## - X2    1   0.05385 7.1614 -26.975
## - X1    1   0.13496 7.2425 -26.637
## - X5    1   0.20709 7.3147 -26.340
## - X6    1   0.45446 7.5620 -25.342
## <none>              7.1076 -25.201
## - X7    1   0.94872 8.0563 -23.442
## - X3    1   1.22165 8.3292 -22.443
## 
## Step:  AIC=-27.2
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7
## 
##        Df Sum of Sq    RSS     AIC
## - X4    1   0.01127 7.1189 -29.154
## - X2    1   0.05390 7.1615 -28.974
## - X1    1   0.13699 7.2446 -28.628
## - X5    1   0.20769 7.3153 -28.337
## - X6    1   0.45911 7.5667 -27.323
## <none>              7.1076 -27.201
## - X7    1   0.95446 8.0621 -25.421
## - X3    1   1.25421 8.3618 -24.326
## 
## Step:  AIC=-29.15
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7
## 
##        Df Sum of Sq     RSS      AIC
## - X2    1    0.1409  7.2598 -30.5654
## - X5    1    0.4878  7.6066 -29.1654
## <none>               7.1189 -29.1535
## - X6    1    1.1498  8.2686 -26.6619
## - X1    1    2.6634  9.7823 -21.6187
## - X7    1    3.8818 11.0007 -18.0972
## - X3    1   17.0862 24.2051   5.5609
## 
## Step:  AIC=-30.57
## log(y) ~ X1 + X3 + X5 + X6 + X7
## 
##        Df Sum of Sq     RSS      AIC
## - X5    1    0.3665  7.6263 -31.0878
## <none>               7.2598 -30.5654
## - X6    1    1.1128  8.3726 -28.2870
## - X1    1    2.6550  9.9148 -23.2150
## - X7    1    4.5672 11.8270 -17.9245
## - X3    1   19.3750 26.6348   6.4306
## 
## Step:  AIC=-31.09
## log(y) ~ X1 + X3 + X6 + X7
## 
##        Df Sum of Sq    RSS     AIC
## <none>               7.626 -31.088
## - X6    1    1.5290  9.155 -27.606
## - X1    1    2.7319 10.358 -23.902
## - X7    1    4.8252 12.452 -18.381
## - X3    1   26.0905 33.717  11.504
ols_vif_tol(model_wf_rm8_aic_log)
## # A tibble: 4 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.281  3.56
## 2 X3            0.773  1.29
## 3 X6            0.949  1.05
## 4 X7            0.252  3.96
# remove X9
model_wf_rm9_log <- lm(log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8, data=table_wf)
ols_vif_tol(model_wf_rm9_log)
## # A tibble: 8 x 3
##   Variables Tolerance    VIF
##   <chr>         <dbl>  <dbl>
## 1 X1          0.0104   95.8 
## 2 X2          0.134     7.48
## 3 X3          0.0341   29.3 
## 4 X4          0.00998 100.  
## 5 X5          0.113     8.83
## 6 X6          0.433     2.31
## 7 X7          0.0495   20.2 
## 8 X8          0.919     1.09
model_wf_rm9_aic_log <- stepAIC(model_wf_rm9_log)
## Start:  AIC=-28.16
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8
## 
##        Df Sum of Sq    RSS     AIC
## - X4    1   0.00125 6.4404 -30.158
## - X2    1   0.03429 6.4735 -30.005
## - X5    1   0.14939 6.5886 -29.476
## - X1    1   0.15258 6.5918 -29.461
## <none>              6.4392 -28.164
## - X6    1   0.59615 7.0353 -27.508
## - X8    1   0.66842 7.1076 -27.201
## - X7    1   0.99933 7.4385 -25.836
## - X3    1   1.34802 7.7872 -24.462
## 
## Step:  AIC=-30.16
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8
## 
##        Df Sum of Sq     RSS     AIC
## - X2    1    0.0670  6.5074 -31.848
## - X5    1    0.2880  6.7284 -30.846
## <none>               6.4404 -30.158
## - X8    1    0.6784  7.1189 -29.153
## - X6    1    1.2983  7.7387 -26.649
## - X1    1    2.0749  8.5153 -23.780
## - X7    1    3.5965 10.0369 -18.848
## - X3    1   16.2255 22.6659   5.590
## 
## Step:  AIC=-31.85
## log(y) ~ X1 + X3 + X5 + X6 + X7 + X8
## 
##        Df Sum of Sq     RSS     AIC
## - X5    1    0.2255  6.7330 -32.825
## <none>               6.5074 -31.848
## - X8    1    0.7523  7.2598 -30.565
## - X6    1    1.2782  7.7856 -28.468
## - X1    1    2.1911  8.6986 -25.141
## - X7    1    4.5538 11.0612 -17.933
## - X3    1   18.9765 25.4839   7.105
## 
## Step:  AIC=-32.83
## log(y) ~ X1 + X3 + X6 + X7 + X8
## 
##        Df Sum of Sq    RSS     AIC
## <none>               6.733 -32.825
## - X8    1    0.8934  7.626 -31.088
## - X6    1    1.6647  8.398 -28.197
## - X1    1    2.4940  9.227 -25.372
## - X7    1    4.7585 11.491 -18.788
## - X3    1   26.5284 33.261  13.096
ols_vif_tol(model_wf_rm9_aic_log)
## # A tibble: 5 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.280  3.58
## 2 X3            0.771  1.30
## 3 X6            0.945  1.06
## 4 X7            0.252  3.97
## 5 X8            0.963  1.04
# remove X6
model_wf_rm6_log <- lm(log(y) ~ (X1 +X2 + X3+ X4 + X5 + X7 + X8 + X9), data=table_wf)
ols_vif_tol(model_wf_rm6_log)
## # A tibble: 8 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1           0.0162 61.5 
## 2 X2           0.167   6.00
## 3 X3           0.0563 17.8 
## 4 X4           0.0182 54.8 
## 5 X5           0.116   8.64
## 6 X7           0.0832 12.0 
## 7 X8           0.182   5.49
## 8 X9           0.174   5.75
model_wf_rm6_aic_log <- stepAIC(model_wf_rm6_log)
## Start:  AIC=-39.8
## log(y) ~ (X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9)
## 
##        Df Sum of Sq    RSS     AIC
## - X3    1    0.0564 4.4250 -41.418
## - X2    1    0.1043 4.4729 -41.095
## - X7    1    0.1349 4.5035 -40.891
## - X5    1    0.1543 4.5229 -40.762
## <none>              4.3686 -39.803
## - X1    1    0.4338 4.8023 -38.963
## - X4    1    1.7475 6.1160 -31.708
## - X9    1    2.6668 7.0353 -27.508
## - X8    1    3.1935 7.5620 -25.342
## 
## Step:  AIC=-41.42
## log(y) ~ X1 + X2 + X4 + X5 + X7 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## - X7    1    0.0848  4.5098 -42.849
## - X2    1    0.3043  4.7293 -41.423
## <none>               4.4250 -41.418
## - X5    1    0.9642  5.3892 -37.504
## - X9    1    3.3632  7.7882 -26.458
## - X8    1    3.9187  8.3437 -24.391
## - X1    1    4.6412  9.0662 -21.899
## - X4    1   16.0173 20.4423   2.492
## 
## Step:  AIC=-42.85
## log(y) ~ X1 + X2 + X4 + X5 + X8 + X9
## 
##        Df Sum of Sq     RSS     AIC
## <none>               4.5098 -42.849
## - X2    1    0.6110  5.1208 -41.037
## - X5    1    0.8871  5.3969 -39.461
## - X9    1    3.2799  7.7896 -28.452
## - X8    1    3.8347  8.3444 -26.388
## - X1    1    5.0057  9.5155 -22.448
## - X4    1   15.9600 20.4698   0.533
ols_vif_tol(model_wf_rm6_aic_log)
## # A tibble: 6 x 3
##   Variables Tolerance   VIF
##   <chr>         <dbl> <dbl>
## 1 X1            0.119  8.39
## 2 X2            0.313  3.19
## 3 X4            0.123  8.12
## 4 X5            0.343  2.92
## 5 X8            0.209  4.79
## 6 X9            0.205  4.88
library(huxtable)
huxreg(model_wf_rm1_log, model_wf_rm2_log, model_wf_rm3_log, model_wf_rm4_log, model_wf_rm5_log, model_wf_rm6_log, model_wf_rm7_log, model_wf_rm8_log, model_wf_rm9_log, model_wf_full_log)
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
(Intercept) 3.280     3.703     7.731 *** 1.200     2.581 *** 5.523     7.487 **  -0.182  -0.127   3.402    
(1.690)    (2.333)    (1.678)    (2.110)    (0.547)    (3.075)    (2.314)    (4.072) (3.844)  (3.150)   
X2 -1.243              6.526     -5.001     -2.144     4.655     8.550     -3.730  -2.980   -1.024    
(5.025)             (5.358)    (5.568)    (5.445)    (6.574)    (4.819)    (9.350) (8.912)  (6.995)   
X3 0.183 *** 0.167 *            0.278 *** 0.201 **  0.046     0.002     0.277  0.287 * 0.178    
(0.034)    (0.080)             (0.032)    (0.067)    (0.088)    (0.057)    (0.146) (0.137)  (0.111)   
X4 0.104 **  0.119     0.286 ***          0.088     0.253 **  0.284 *** 0.027  0.009   0.109    
(0.032)    (0.090)    (0.035)             (0.084)    (0.087)    (0.066)    (0.153) (0.143)  (0.115)   
X5 -0.008     -0.013     -0.055 *   0.013              -0.031     -0.050     0.036  0.031   -0.010    
(0.021)    (0.028)    (0.023)    (0.027)             (0.036)    (0.030)    (0.047) (0.044)  (0.036)   
X6 -0.396 *   -0.375     -0.161     -0.531 **  -0.408              -0.138     -0.335  -0.385   -0.389    
(0.164)    (0.188)    (0.168)    (0.155)    (0.199)             (0.174)    (0.289) (0.276)  (0.216)   
X7 4.317 **  3.975 *   0.959     6.088 *** 4.611 *   1.517              5.230  5.352   4.233    
(1.466)    (1.495)    (1.178)    (1.266)    (1.814)    (1.884)             (3.124) (2.965)  (2.339)   
X8 0.629 *** 0.632 *** 0.680 *** 0.606 *** 0.618 *** 0.614 *** 0.657 ***       0.125   0.630 ***
(0.142)    (0.145)    (0.151)    (0.147)    (0.139)    (0.157)    (0.156)          (0.085)  (0.149)   
X9 -0.461 *** -0.464 *** -0.513 *** -0.436 **  -0.453 *** -0.461 **  -0.490 *** 0.001         -0.462 ** 
(0.116)    (0.119)    (0.122)    (0.119)    (0.114)    (0.129)    (0.128)    (0.073)        (0.122)   
X1          -0.042     -0.457 *** 0.250 **  0.048     -0.345     -0.418 *   0.242  0.255   -0.014    
         (0.210)    (0.096)    (0.083)    (0.173)    (0.239)    (0.197)    (0.383) (0.362)  (0.292)   
N 30         30         30         30         30         30         30         30      30       30        
R2 0.947     0.947     0.941     0.945     0.947     0.939     0.939     0.900  0.910   0.947    
logLik -11.407     -11.422     -13.216     -12.059     -11.458     -13.667     -13.681     -20.968  -19.486   -11.406    
AIC 42.815     42.843     46.432     44.118     42.916     47.333     47.361     61.935  58.972   44.811    
*** p < 0.001; ** p < 0.01; * p < 0.05.
huxreg(model_wf_rm1_aic_log, model_wf_rm2_aic_log, model_wf_rm3_aic_log, model_wf_rm4_aic_log, model_wf_rm5_aic_log, model_wf_rm6_aic_log, model_wf_rm7_aic_log, model_wf_rm8_aic_log, model_wf_rm9_aic_log, model_wf_aic_log)
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
(Intercept) 2.692 *** 2.692 *** 6.882 *** 2.307 *** 2.692 *** 6.882 *** 6.882 *** 2.587 *** 2.225 *** 2.692 ***
(0.445)    (0.445)    (1.432)    (0.410)    (0.445)    (1.432)    (1.432)    (0.494)    (0.515)    (0.445)   
X3 0.184 *** 0.184 ***          0.263 *** 0.184 ***                   0.266 *** 0.268 *** 0.184 ***
(0.032)    (0.032)             (0.022)    (0.032)                      (0.029)    (0.028)    (0.032)   
X4 0.109 *** 0.109 *** 0.294 ***          0.109 *** 0.294 *** 0.294 ***                   0.109 ***
(0.026)    (0.026)    (0.033)             (0.026)    (0.033)    (0.033)                      (0.026)   
X6 -0.368 *   -0.368 *            -0.532 **  -0.368 *                     -0.416 *   -0.435 *   -0.368 *  
(0.146)    (0.146)             (0.144)    (0.146)                      (0.186)    (0.179)    (0.146)   
X7 4.085 **  4.085 **           5.453 *** 4.085 **                    5.209 *** 5.174 *** 4.085 ** 
(1.213)    (1.213)             (1.002)    (1.213)                      (1.310)    (1.256)    (1.213)   
X8 0.612 *** 0.612 *** 0.629 *** 0.613 *** 0.612 *** 0.629 *** 0.629 ***          0.141     0.612 ***
(0.133)    (0.133)    (0.142)    (0.137)    (0.133)    (0.142)    (0.142)             (0.079)    (0.133)   
X9 -0.448 *** -0.448 *** -0.471 *** -0.433 *** -0.448 *** -0.471 *** -0.471 ***                   -0.448 ***
(0.108)    (0.108)    (0.115)    (0.112)    (0.108)    (0.115)    (0.115)                      (0.108)   
X1                   -0.432 *** 0.207 ***          -0.432 *** -0.432 *** 0.208 **  0.199 **          
                  (0.086)    (0.053)             (0.086)    (0.086)    (0.070)    (0.067)            
X2                   8.217                       8.217     8.217                               
                  (4.655)                      (4.655)    (4.655)                              
X5                   -0.043 *                     -0.043 *   -0.043 *                             
                  (0.020)                      (0.020)    (0.020)                              
N 30         30         30         30         30         30         30         30         30         30        
R2 0.947     0.947     0.937     0.943     0.947     0.937     0.937     0.893     0.906     0.947    
logLik -11.581     -11.581     -14.144     -12.652     -11.581     -14.144     -14.144     -22.024     -20.155     -11.581    
AIC 39.161     39.161     44.288     41.305     39.161     44.288     44.288     56.049     54.311     39.161    
*** p < 0.001; ** p < 0.01; * p < 0.05.

(d) (3) Variable selection

# Interaction regression for full
model_wf_full_log_inter <- lm(log(y)~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_aic_log_inter <- stepAIC(model_wf_full_log_inter)
## Start:  AIC=-86.68
## log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + 
##     X4:X9 + X5:X6 + X5:X7 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + 
##     X4:X9 + X5:X6 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X7 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X6 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X6 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X4 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X8 + 
##     X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X5 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X4 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-86.68
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X8  1   0.00125 0.27702 -88.546
## - X3:X9  1   0.00166 0.27743 -88.501
## - X1:X9  1   0.00171 0.27748 -88.496
## - X4:X9  1   0.00224 0.27801 -88.439
## - X5:X8  1   0.00375 0.27952 -88.276
## - X3:X8  1   0.01365 0.28942 -87.232
## - X5:X9  1   0.01394 0.28971 -87.202
## <none>               0.27577 -86.682
## - X4:X8  1   0.01926 0.29503 -86.656
## - X8:X9  1   0.02380 0.29957 -86.198
## - X1:X2  1   0.02492 0.30069 -86.086
## - X2:X8  1   0.02521 0.30098 -86.057
## - X6:X8  1   0.02975 0.30552 -85.608
## - X6:X9  1   0.03024 0.30601 -85.560
## - X2:X9  1   0.03404 0.30981 -85.190
## - X7:X8  1   0.04050 0.31627 -84.570
## - X7:X9  1   0.08581 0.36158 -80.554
## - X1:X3  1   1.65959 1.93536 -30.227
## 
## Step:  AIC=-88.55
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X9  1   0.00103 0.27806 -90.434
## - X5:X8  1   0.00630 0.28332 -89.871
## - X3:X9  1   0.01467 0.29170 -88.997
## <none>               0.27702 -88.546
## - X5:X9  1   0.01990 0.29692 -88.465
## - X2:X8  1   0.02658 0.30360 -87.797
## - X1:X2  1   0.02706 0.30408 -87.750
## - X3:X8  1   0.02955 0.30658 -87.504
## - X6:X9  1   0.03164 0.30866 -87.301
## - X6:X8  1   0.03412 0.31114 -87.061
## - X4:X8  1   0.03459 0.31162 -87.015
## - X2:X9  1   0.03623 0.31325 -86.859
## - X8:X9  1   0.03768 0.31470 -86.720
## - X7:X8  1   0.04267 0.31969 -86.248
## - X1:X9  1   0.08036 0.35738 -82.904
## - X7:X9  1   0.08949 0.36652 -82.147
## - X1:X3  1   1.67908 1.95611 -31.907
## 
## Step:  AIC=-90.43
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.01021 0.28826 -91.352
## <none>               0.27806 -90.434
## - X3:X9  1   0.01962 0.29768 -90.388
## - X1:X2  1   0.02775 0.30580 -89.580
## - X3:X8  1   0.02855 0.30661 -89.502
## - X5:X9  1   0.02886 0.30692 -89.471
## - X6:X9  1   0.03330 0.31136 -89.040
## - X8:X9  1   0.03752 0.31557 -88.637
## - X6:X8  1   0.04082 0.31888 -88.324
## - X7:X8  1   0.05026 0.32832 -87.449
## - X2:X8  1   0.07559 0.35365 -85.220
## - X1:X9  1   0.08639 0.36445 -84.317
## - X2:X9  1   0.09477 0.37282 -83.635
## - X7:X9  1   0.09547 0.37353 -83.579
## - X4:X8  1   0.11185 0.38991 -82.291
## - X1:X3  1   1.76157 2.03963 -32.653
## 
## Step:  AIC=-91.35
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.01387 0.30213 -91.943
## <none>               0.28826 -91.352
## - X1:X2  1   0.02606 0.31432 -90.756
## - X6:X9  1   0.02892 0.31718 -90.484
## - X5:X9  1   0.03517 0.32343 -89.899
## - X6:X8  1   0.03526 0.32352 -89.891
## - X8:X9  1   0.03647 0.32474 -89.778
## - X7:X8  1   0.05418 0.34244 -88.186
## - X3:X8  1   0.06678 0.35505 -87.101
## - X1:X9  1   0.08233 0.37059 -85.815
## - X2:X8  1   0.09026 0.37852 -85.180
## - X4:X8  1   0.11594 0.40420 -83.211
## - X2:X9  1   0.12196 0.41023 -82.767
## - X7:X9  1   0.19579 0.48405 -77.803
## - X1:X3  1   1.79585 2.08412 -34.006
## 
## Step:  AIC=-91.94
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.01568 0.31781 -92.425
## <none>               0.30213 -91.943
## - X6:X8  1   0.02200 0.32413 -91.834
## - X5:X9  1   0.02250 0.32464 -91.788
## - X1:X2  1   0.03453 0.33666 -90.696
## - X7:X8  1   0.04139 0.34352 -90.092
## - X8:X9  1   0.05081 0.35295 -89.279
## - X1:X9  1   0.07027 0.37241 -87.669
## - X2:X8  1   0.07640 0.37853 -87.180
## - X3:X8  1   0.09504 0.39717 -85.737
## - X4:X8  1   0.10557 0.40770 -84.952
## - X2:X9  1   0.10898 0.41111 -84.703
## - X7:X9  1   0.20828 0.51041 -78.212
## - X1:X3  1   1.80109 2.10322 -35.732
## 
## Step:  AIC=-92.43
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X6:X8 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X8  1   0.00642 0.32423 -93.825
## - X1:X2  1   0.01993 0.33773 -92.601
## <none>               0.31781 -92.425
## - X5:X9  1   0.02449 0.34230 -92.198
## - X7:X8  1   0.02573 0.34354 -92.090
## - X8:X9  1   0.03582 0.35363 -91.221
## - X1:X9  1   0.06065 0.37846 -89.186
## - X2:X8  1   0.06122 0.37903 -89.140
## - X3:X8  1   0.08481 0.40262 -87.329
## - X4:X8  1   0.09252 0.41033 -86.760
## - X2:X9  1   0.09419 0.41200 -86.638
## - X7:X9  1   0.23406 0.55187 -77.869
## - X1:X3  1   1.89418 2.21199 -36.219
## 
## Step:  AIC=-93.83
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X7:X8  1   0.02050 0.34473 -93.986
## - X1:X2  1   0.02195 0.34617 -93.860
## <none>               0.32423 -93.825
## - X6     1   0.02936 0.35359 -93.225
## - X5:X9  1   0.03666 0.36089 -92.611
## - X8:X9  1   0.05990 0.38412 -90.740
## - X2:X8  1   0.10202 0.42624 -87.618
## - X2:X9  1   0.16870 0.49293 -83.258
## - X7:X9  1   0.23395 0.55817 -79.529
## - X1:X9  1   0.25322 0.57745 -78.510
## - X3:X8  1   0.26381 0.58803 -77.965
## - X4:X8  1   0.40165 0.72587 -71.647
## - X1:X3  1   1.90127 2.22550 -38.036
## 
## Step:  AIC=-93.99
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + 
##     X1:X3 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X7:X9 + 
##     X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X2  1   0.01744 0.36216 -94.506
## <none>               0.34473 -93.986
## - X6     1   0.02380 0.36853 -93.983
## - X8:X9  1   0.04553 0.39026 -92.264
## - X5:X9  1   0.04913 0.39386 -91.988
## - X2:X8  1   0.08418 0.42891 -89.432
## - X2:X9  1   0.15080 0.49553 -85.100
## - X1:X9  1   0.27457 0.61930 -78.411
## - X3:X8  1   0.30355 0.64828 -77.039
## - X7:X9  1   0.32337 0.66809 -76.136
## - X4:X8  1   0.42623 0.77096 -71.840
## - X1:X3  1   1.88785 2.23258 -39.941
## 
## Step:  AIC=-94.51
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X9 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6     1   0.02085 0.38302 -94.826
## <none>               0.36216 -94.506
## - X8:X9  1   0.03703 0.39920 -93.585
## - X5:X9  1   0.03720 0.39936 -93.573
## - X2:X8  1   0.06882 0.43098 -91.286
## - X2:X9  1   0.13369 0.49585 -87.080
## - X1:X9  1   0.26000 0.62217 -80.272
## - X3:X8  1   0.29045 0.65261 -78.839
## - X7:X9  1   0.30630 0.66847 -78.119
## - X4:X8  1   0.40893 0.77109 -73.834
## - X1:X3  1   1.87267 2.23483 -41.911
## 
## Step:  AIC=-94.83
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9 + X1:X3 + X1:X9 + 
##     X2:X8 + X2:X9 + X3:X8 + X4:X8 + X5:X9 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X9  1   0.02376 0.40678 -95.021
## - X8:X9  1   0.02579 0.40880 -94.871
## <none>               0.38302 -94.826
## - X2:X8  1   0.04986 0.43287 -93.155
## - X2:X9  1   0.11290 0.49591 -89.077
## - X1:X9  1   0.23967 0.62268 -82.247
## - X3:X8  1   0.28141 0.66443 -80.301
## - X7:X9  1   0.28680 0.66981 -80.059
## - X4:X8  1   0.39670 0.77972 -75.501
## - X1:X3  1   2.28557 2.66859 -38.589
## 
## Step:  AIC=-95.02
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9 + X1:X3 + X1:X9 + 
##     X2:X8 + X2:X9 + X3:X8 + X4:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.02190 0.42867 -95.448
## <none>               0.40678 -95.021
## - X2:X8  1   0.02957 0.43635 -94.916
## - X2:X9  1   0.08976 0.49654 -91.039
## - X5     1   0.18602 0.59280 -85.723
## - X7:X9  1   0.26382 0.67060 -82.024
## - X1:X9  1   0.32901 0.73579 -79.240
## - X3:X8  1   0.35450 0.76128 -78.219
## - X4:X8  1   0.43472 0.84149 -75.213
## - X1:X3  1   2.29575 2.70253 -40.210
## 
## Step:  AIC=-95.45
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9 + X1:X3 + X1:X9 + 
##     X2:X8 + X2:X9 + X3:X8 + X4:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.01037 0.43904 -96.731
## <none>               0.42867 -95.448
## - X2:X9  1   0.06896 0.49763 -92.973
## - X5     1   0.17518 0.60385 -87.169
## - X7:X9  1   0.24199 0.67066 -84.021
## - X1:X9  1   0.30722 0.73589 -81.236
## - X3:X8  1   0.34794 0.77661 -79.620
## - X4:X8  1   0.47278 0.90146 -75.148
## - X1:X3  1   2.41556 2.84423 -40.677
## 
## Step:  AIC=-96.73
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9 + X1:X3 + X1:X9 + 
##     X2:X9 + X3:X8 + X4:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.43904 -96.731
## - X5     1   0.17280 0.61185 -88.774
## - X7:X9  1   0.24527 0.68431 -85.416
## - X2:X9  1   0.30033 0.73937 -83.095
## - X1:X9  1   0.31184 0.75088 -82.631
## - X3:X8  1   0.40524 0.84428 -79.114
## - X4:X8  1   0.66804 1.10708 -70.984
## - X1:X3  1   2.49992 2.93897 -41.694
# Interaction regression for remove X1
model_wf_rm1_log_inter <- lm(log(y) ~ (X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_rm1_aic_log_inter <- stepAIC(model_wf_rm1_log_inter)
## Start:  AIC=-82.9
## log(y) ~ (X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X7 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X7 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X6 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-82.9
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X9  1   0.00112 0.35850 -84.810
## - X5:X8  1   0.00181 0.35920 -84.753
## - X6:X9  1   0.00316 0.36055 -84.640
## - X3:X9  1   0.00489 0.36228 -84.496
## - X4:X9  1   0.00706 0.36445 -84.317
## - X2:X8  1   0.00992 0.36730 -84.083
## - X8:X9  1   0.01084 0.36822 -84.008
## - X4:X8  1   0.01150 0.36888 -83.954
## - X3:X8  1   0.01215 0.36953 -83.902
## - X2:X9  1   0.01274 0.37013 -83.853
## - X6:X8  1   0.01862 0.37601 -83.380
## <none>               0.35738 -82.904
## - X7:X8  1   0.02483 0.38221 -82.889
## - X7:X9  1   0.03695 0.39433 -81.953
## - X2:X4  1   0.47386 0.83125 -59.581
## - X2:X3  1   0.59876 0.95614 -55.381
## - X2:X5  1   0.82454 1.18192 -49.022
## 
## Step:  AIC=-84.81
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.00201 0.36051 -86.643
## - X6:X9  1   0.00228 0.36078 -86.620
## - X3:X9  1   0.00578 0.36428 -86.331
## - X8:X9  1   0.01166 0.37016 -85.850
## - X4:X9  1   0.01246 0.37096 -85.786
## - X3:X8  1   0.01435 0.37285 -85.633
## - X4:X8  1   0.02423 0.38274 -84.848
## <none>               0.35850 -84.810
## - X2:X8  1   0.02892 0.38742 -84.483
## - X6:X8  1   0.03402 0.39252 -84.091
## - X2:X9  1   0.04290 0.40141 -83.419
## - X7:X8  1   0.04659 0.40509 -83.145
## - X7:X9  1   0.06798 0.42649 -81.601
## - X2:X4  1   0.48258 0.84109 -61.228
## - X2:X3  1   0.63229 0.99079 -56.313
## - X2:X5  1   1.15133 1.50983 -43.676
## 
## Step:  AIC=-86.64
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.00156 0.36207 -88.513
## - X3:X9  1   0.00441 0.36492 -88.278
## - X8:X9  1   0.00971 0.37022 -87.845
## - X4:X9  1   0.01064 0.37115 -87.770
## - X3:X8  1   0.01289 0.37340 -87.589
## <none>               0.36051 -86.643
## - X4:X8  1   0.02581 0.38632 -86.568
## - X2:X8  1   0.02997 0.39048 -86.247
## - X6:X8  1   0.03329 0.39380 -85.993
## - X7:X8  1   0.04479 0.40530 -85.130
## - X2:X9  1   0.04652 0.40703 -85.002
## - X7:X9  1   0.06720 0.42771 -83.515
## - X2:X4  1   0.48112 0.84163 -63.208
## - X2:X3  1   0.63947 0.99998 -58.037
## - X2:X5  1   1.14951 1.51002 -45.672
## 
## Step:  AIC=-88.51
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.00942 0.37149 -89.743
## - X3:X9  1   0.01266 0.37473 -89.482
## - X4:X9  1   0.01702 0.37909 -89.135
## <none>               0.36207 -88.513
## - X3:X8  1   0.02767 0.38974 -88.304
## - X2:X8  1   0.02898 0.39105 -88.203
## - X4:X8  1   0.03665 0.39872 -87.621
## - X2:X9  1   0.04561 0.40768 -86.954
## - X7:X8  1   0.09991 0.46198 -83.203
## - X7:X9  1   0.13698 0.49905 -80.887
## - X6:X8  1   0.32694 0.68901 -71.211
## - X2:X4  1   0.51434 0.87641 -63.993
## - X2:X3  1   0.63947 1.00154 -59.990
## - X2:X5  1   1.16650 1.52857 -47.306
## 
## Step:  AIC=-89.74
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + 
##     X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.00629 0.37778 -91.239
## - X4:X9  1   0.01221 0.38370 -90.773
## - X3:X8  1   0.01842 0.38991 -90.291
## - X2:X8  1   0.01959 0.39108 -90.201
## <none>               0.37149 -89.743
## - X4:X8  1   0.02784 0.39933 -89.575
## - X2:X9  1   0.03636 0.40786 -88.941
## - X7:X8  1   0.09297 0.46446 -85.042
## - X7:X9  1   0.13179 0.50328 -82.634
## - X6:X8  1   0.32177 0.69326 -73.027
## - X2:X4  1   0.56066 0.93215 -64.144
## - X2:X3  1   0.70382 1.07531 -59.858
## - X2:X5  1   1.17679 1.54828 -48.922
## 
## Step:  AIC=-91.24
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + X7:X8 + 
##     X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X9  1   0.00777 0.38554 -92.629
## - X2:X8  1   0.01395 0.39173 -92.152
## <none>               0.37778 -91.239
## - X3:X8  1   0.02828 0.40606 -91.074
## - X2:X9  1   0.03475 0.41252 -90.600
## - X4:X8  1   0.03705 0.41482 -90.433
## - X7:X8  1   0.13183 0.50961 -84.259
## - X7:X9  1   0.28304 0.66082 -76.464
## - X6:X8  1   0.31554 0.69332 -75.024
## - X2:X4  1   0.57053 0.94830 -65.628
## - X2:X3  1   0.71720 1.09497 -61.314
## - X2:X5  1   1.17894 1.55672 -50.758
## 
## Step:  AIC=-92.63
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.00620 0.39175 -94.150
## <none>               0.38554 -92.629
## - X3:X8  1   0.02925 0.41479 -92.435
## - X2:X9  1   0.03198 0.41753 -92.238
## - X4:X8  1   0.07540 0.46094 -89.270
## - X7:X8  1   0.13553 0.52107 -85.592
## - X7:X9  1   0.28341 0.66895 -78.097
## - X6:X8  1   0.31350 0.69904 -76.777
## - X2:X3  1   0.71102 1.09657 -63.270
## - X2:X4  1   0.74155 1.12709 -62.447
## - X2:X5  1   1.25071 1.63625 -51.264
## 
## Step:  AIC=-94.15
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X9 + X3:X8 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X8  1   0.02406 0.41581 -94.362
## <none>               0.39175 -94.150
## - X4:X8  1   0.07287 0.46462 -91.032
## - X7:X8  1   0.13519 0.52694 -87.256
## - X2:X9  1   0.13580 0.52754 -87.222
## - X6:X8  1   0.30834 0.70009 -78.732
## - X7:X9  1   0.33742 0.72917 -77.511
## - X2:X3  1   0.73894 1.13069 -64.351
## - X2:X4  1   0.74811 1.13986 -64.109
## - X2:X5  1   1.24528 1.63703 -53.249
## 
## Step:  AIC=-94.36
## log(y) ~ X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X2:X3 + X2:X4 + 
##     X2:X5 + X2:X9 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.41581 -94.362
## - X4:X8  1   0.04971 0.46552 -92.974
## - X2:X9  1   0.11751 0.53332 -88.895
## - X7:X8  1   0.26340 0.67921 -81.641
## - X7:X9  1   0.34302 0.75883 -78.315
## - X6:X8  1   0.36050 0.77631 -77.632
## - X2:X3  1   0.72524 1.14105 -66.077
## - X2:X4  1   0.74088 1.15669 -65.669
## - X2:X5  1   1.22744 1.64325 -55.136
# Interaction regression for remove X4
model_wf_rm4_log_inter <- lm(log(y) ~ (X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_rm4_aic_log_inter <- stepAIC(model_wf_rm4_log_inter)
## Start:  AIC=-80.33
## log(y) ~ (X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X5:X6 + X5:X7 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X5:X6 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X5 + X3:X6 + X3:X8 + X3:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X5 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X6 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X5 + X2:X8 + 
##     X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X8 + X2:X9 + 
##     X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + 
##     X3:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-80.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X9  1   0.00005 0.38941 -82.330
## - X2:X8  1   0.00029 0.38965 -82.311
## - X1:X8  1   0.00055 0.38991 -82.291
## - X1:X2  1   0.00059 0.38995 -82.288
## - X6:X9  1   0.00061 0.38997 -82.287
## - X5:X8  1   0.00071 0.39007 -82.279
## - X2:X9  1   0.00097 0.39033 -82.259
## - X5:X9  1   0.00117 0.39053 -82.243
## - X3:X8  1   0.00199 0.39135 -82.181
## - X3:X9  1   0.00220 0.39156 -82.164
## - X8:X9  1   0.00232 0.39168 -82.155
## - X6:X8  1   0.00871 0.39807 -81.670
## - X7:X8  1   0.01370 0.40306 -81.296
## - X7:X9  1   0.02001 0.40937 -80.830
## <none>               0.38936 -80.333
## - X1:X3  1   0.49263 0.88199 -57.803
## - X1:X5  1   0.54062 0.92999 -56.213
## 
## Step:  AIC=-82.33
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X5 + X1:X8 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X2  1   0.00059 0.39000 -84.284
## - X6:X9  1   0.00067 0.39008 -84.278
## - X2:X8  1   0.00091 0.39032 -84.260
## - X8:X9  1   0.00232 0.39173 -84.151
## - X3:X9  1   0.00398 0.39339 -84.025
## - X3:X8  1   0.00419 0.39360 -84.009
## - X2:X9  1   0.00553 0.39494 -83.906
## - X5:X8  1   0.00607 0.39548 -83.865
## - X5:X9  1   0.00736 0.39677 -83.768
## - X6:X8  1   0.00869 0.39810 -83.667
## - X7:X8  1   0.01406 0.40347 -83.266
## - X7:X9  1   0.02022 0.40963 -82.811
## - X1:X8  1   0.02067 0.41008 -82.778
## <none>               0.38941 -82.330
## - X1:X3  1   0.49364 0.88305 -59.767
## - X1:X5  1   0.54419 0.93360 -58.097
## 
## Step:  AIC=-84.28
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.00055 0.39056 -86.241
## - X2:X8  1   0.00137 0.39137 -86.179
## - X8:X9  1   0.00378 0.39379 -85.994
## - X3:X9  1   0.00478 0.39478 -85.918
## - X3:X8  1   0.00514 0.39514 -85.891
## - X5:X8  1   0.00629 0.39629 -85.804
## - X2:X9  1   0.00654 0.39654 -85.785
## - X5:X9  1   0.00782 0.39782 -85.688
## - X6:X8  1   0.00924 0.39924 -85.582
## - X7:X8  1   0.01474 0.40474 -85.171
## - X7:X9  1   0.02054 0.41055 -84.744
## - X1:X8  1   0.02373 0.41373 -84.512
## <none>               0.39000 -84.284
## - X1:X3  1   0.66803 1.05804 -56.343
## - X1:X5  1   1.14671 1.53671 -45.147
## 
## Step:  AIC=-86.24
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.00333 0.39388 -87.987
## - X2:X8  1   0.00549 0.39604 -87.823
## - X5:X8  1   0.00733 0.39788 -87.684
## - X3:X9  1   0.00839 0.39895 -87.603
## - X3:X8  1   0.00928 0.39984 -87.537
## - X5:X9  1   0.00945 0.40001 -87.524
## - X2:X9  1   0.01894 0.40949 -86.821
## - X1:X8  1   0.02356 0.41411 -86.484
## <none>               0.39056 -86.241
## - X7:X8  1   0.10010 0.49066 -81.396
## - X7:X9  1   0.11310 0.50365 -80.612
## - X6:X8  1   0.29229 0.68285 -71.480
## - X1:X3  1   0.69717 1.08772 -57.513
## - X1:X5  1   1.24983 1.64039 -45.188
## 
## Step:  AIC=-87.99
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.00278 0.39667 -89.776
## - X3:X9  1   0.00555 0.39943 -89.567
## - X5:X9  1   0.00698 0.40086 -89.460
## - X5:X8  1   0.00705 0.40093 -89.455
## - X3:X8  1   0.00742 0.40130 -89.428
## - X2:X9  1   0.01562 0.40950 -88.821
## - X1:X8  1   0.02111 0.41499 -88.421
## <none>               0.39388 -87.987
## - X7:X8  1   0.09678 0.49066 -83.396
## - X7:X9  1   0.11277 0.50665 -82.434
## - X6:X8  1   0.29068 0.68456 -73.405
## - X1:X3  1   0.70415 1.09803 -59.230
## - X1:X5  1   1.29122 1.68510 -46.381
## 
## Step:  AIC=-89.78
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X8 + X5:X9 + X6:X8 + X7:X8 + 
##     X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.01430 0.41097 -90.713
## - X3:X9  1   0.01475 0.41142 -90.680
## - X5:X9  1   0.01476 0.41143 -90.680
## - X3:X8  1   0.01553 0.41219 -90.624
## - X1:X8  1   0.01921 0.41588 -90.357
## <none>               0.39667 -89.776
## - X2:X9  1   0.09374 0.49041 -85.411
## - X7:X8  1   0.11541 0.51207 -84.115
## - X7:X9  1   0.15777 0.55443 -81.730
## - X6:X8  1   0.32324 0.71990 -73.895
## - X1:X3  1   0.71299 1.10965 -60.915
## - X1:X5  1   1.29256 1.68922 -48.308
## 
## Step:  AIC=-90.71
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X9 + X3:X8 + X3:X9 + X5:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X9  1   0.00271 0.41367 -92.516
## - X3:X8  1   0.00314 0.41411 -92.485
## - X3:X9  1   0.00338 0.41434 -92.468
## <none>               0.41097 -90.713
## - X1:X8  1   0.04629 0.45725 -89.512
## - X2:X9  1   0.08706 0.49803 -86.949
## - X7:X8  1   0.15354 0.56450 -83.190
## - X7:X9  1   0.22187 0.63284 -79.762
## - X6:X8  1   0.31324 0.72421 -75.716
## - X1:X3  1   0.71167 1.12264 -62.565
## - X1:X5  1   1.28571 1.69667 -50.176
## 
## Step:  AIC=-92.52
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X9 + X3:X8 + X3:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.00164 0.41532 -94.397
## - X3:X8  1   0.00372 0.41739 -94.248
## <none>               0.41367 -92.516
## - X1:X8  1   0.05948 0.47316 -90.486
## - X2:X9  1   0.10365 0.51733 -87.808
## - X7:X8  1   0.16564 0.57932 -84.413
## - X7:X9  1   0.26580 0.67948 -79.629
## - X6:X8  1   0.43081 0.84448 -73.107
## - X1:X3  1   0.71449 1.12816 -64.418
## - X1:X5  1   1.32495 1.73862 -51.443
## 
## Step:  AIC=-94.4
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X9 + X3:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X8  1   0.00236 0.41767 -96.228
## <none>               0.41532 -94.397
## - X1:X8  1   0.05850 0.47381 -92.444
## - X2:X9  1   0.11345 0.52877 -89.152
## - X7:X8  1   0.16978 0.58510 -86.115
## - X7:X9  1   0.29297 0.70829 -80.383
## - X6:X8  1   0.43209 0.84741 -75.003
## - X1:X3  1   0.71301 1.12833 -66.414
## - X1:X5  1   1.45822 1.87353 -51.201
## 
## Step:  AIC=-96.23
## log(y) ~ X1 + X2 + X3 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X5 + 
##     X1:X8 + X2:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.41767 -96.228
## - X1:X8  1   0.05616 0.47383 -94.443
## - X2:X9  1   0.11497 0.53264 -90.933
## - X7:X8  1   0.18746 0.60513 -87.105
## - X7:X9  1   0.29100 0.70867 -82.367
## - X6:X8  1   0.43114 0.84881 -76.953
## - X1:X3  1   0.72525 1.14292 -68.028
## - X1:X5  1   1.48475 1.90242 -52.742
# Interaction regression for remove X2
model_wf_rm2_log_inter <- lm(log(y) ~ (X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_rm2_aic_log_inter <- stepAIC(model_wf_rm2_log_inter)
## Start:  AIC=-78.4
## log(y) ~ (X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X7 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X6 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X5 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X7 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X6 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X5 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X4 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-78.4
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1  0.003465 0.41871 -80.154
## - X6:X9  1  0.010677 0.42592 -79.641
## - X1:X5  1  0.014768 0.43001 -79.354
## - X4:X9  1  0.026738 0.44198 -78.531
## <none>               0.41524 -78.403
## - X4:X8  1  0.029652 0.44489 -78.334
## - X1:X8  1  0.030218 0.44546 -78.295
## - X1:X9  1  0.034629 0.44987 -78.000
## - X6:X8  1  0.039224 0.45446 -77.695
## - X3:X8  1  0.041783 0.45702 -77.527
## - X3:X9  1  0.042480 0.45772 -77.481
## - X7:X9  1  0.044612 0.45985 -77.341
## - X1:X4  1  0.047302 0.46254 -77.166
## - X7:X8  1  0.050096 0.46534 -76.986
## - X5:X8  1  0.057908 0.47315 -76.486
## - X5:X9  1  0.083547 0.49879 -74.903
## - X1:X3  1  0.089353 0.50459 -74.556
## 
## Step:  AIC=-80.15
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X5 + X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X5  1  0.011341 0.43005 -81.352
## - X6:X9  1  0.017223 0.43593 -80.944
## <none>               0.41871 -80.154
## - X1:X4  1  0.044654 0.46336 -79.114
## - X3:X8  1  0.046860 0.46557 -78.971
## - X4:X8  1  0.047136 0.46584 -78.953
## - X1:X8  1  0.049186 0.46789 -78.822
## - X4:X9  1  0.052682 0.47139 -78.598
## - X3:X9  1  0.054040 0.47275 -78.512
## - X5:X8  1  0.055767 0.47447 -78.402
## - X7:X9  1  0.057586 0.47629 -78.288
## - X1:X9  1  0.062605 0.48131 -77.973
## - X6:X8  1  0.065890 0.48460 -77.769
## - X7:X8  1  0.070815 0.48952 -77.466
## - X5:X9  1  0.080520 0.49923 -76.877
## - X1:X3  1  0.133849 0.55255 -73.832
## 
## Step:  AIC=-81.35
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.01254 0.44259 -82.490
## <none>               0.43005 -81.352
## - X3:X8  1   0.04311 0.47315 -80.486
## - X4:X8  1   0.04621 0.47626 -80.290
## - X1:X8  1   0.04655 0.47659 -80.269
## - X5:X8  1   0.05013 0.48018 -80.044
## - X7:X9  1   0.05057 0.48061 -80.017
## - X4:X9  1   0.05311 0.48316 -79.858
## - X3:X9  1   0.05344 0.48349 -79.838
## - X6:X8  1   0.05798 0.48803 -79.558
## - X1:X9  1   0.06064 0.49068 -79.395
## - X7:X8  1   0.06344 0.49349 -79.224
## - X5:X9  1   0.07875 0.50880 -78.307
## - X1:X4  1   0.24052 0.67057 -70.025
## - X1:X3  1   1.65844 2.08848 -35.943
## 
## Step:  AIC=-82.49
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X8 + X1:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.44259 -82.490
## - X3:X8  1   0.03267 0.47526 -82.353
## - X4:X8  1   0.03382 0.47640 -82.281
## - X1:X8  1   0.03578 0.47836 -82.157
## - X5:X8  1   0.03850 0.48108 -81.987
## - X4:X9  1   0.04058 0.48316 -81.858
## - X3:X9  1   0.04710 0.48968 -81.456
## - X1:X9  1   0.04999 0.49258 -81.279
## - X5:X9  1   0.06913 0.51171 -80.136
## - X7:X9  1   0.07119 0.51377 -80.015
## - X7:X8  1   0.10024 0.54282 -78.365
## - X6:X8  1   0.21954 0.66213 -72.405
## - X1:X4  1   0.24063 0.68322 -71.464
## - X1:X3  1   1.77755 2.22014 -36.109
# Interaction regression for remove X5
model_wf_rm5_log_inter <- lm(log(y) ~ (X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_rm5_aic_log_inter <- stepAIC(model_wf_rm5_log_inter)
## Start:  AIC=-86.61
## log(y) ~ (X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X4:X6 + X4:X7 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X4:X6 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X6 + X3:X7 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X6 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X4 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X6 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X8 + 
##     X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X3 + X2:X8 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X6 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-86.61
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X9  1   0.00177 0.31763 -88.442
## - X3:X9  1   0.00579 0.32165 -88.065
## - X1:X8  1   0.00626 0.32212 -88.021
## - X1:X2  1   0.00700 0.32286 -87.952
## - X4:X9  1   0.00735 0.32321 -87.920
## - X6:X9  1   0.01501 0.33087 -87.217
## - X8:X9  1   0.01624 0.33210 -87.105
## - X4:X8  1   0.01848 0.33434 -86.904
## <none>               0.31586 -86.610
## - X6:X8  1   0.02268 0.33854 -86.529
## - X3:X8  1   0.02502 0.34088 -86.322
## - X7:X8  1   0.03534 0.35120 -85.428
## - X2:X8  1   0.04737 0.36323 -84.418
## - X7:X9  1   0.06520 0.38106 -82.980
## - X2:X9  1   0.06937 0.38523 -82.654
## - X1:X4  1   0.10098 0.41684 -80.287
## - X1:X3  1   0.52152 0.83738 -59.360
## 
## Step:  AIC=-88.44
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.00594 0.32357 -89.886
## - X1:X2  1   0.00823 0.32585 -89.675
## - X6:X9  1   0.01907 0.33670 -88.693
## <none>               0.31763 -88.442
## - X8:X9  1   0.02980 0.34743 -87.752
## - X4:X9  1   0.03006 0.34769 -87.729
## - X1:X8  1   0.04288 0.36051 -86.643
## - X3:X8  1   0.04295 0.36058 -86.637
## - X6:X8  1   0.04601 0.36363 -86.384
## - X2:X8  1   0.05438 0.37201 -85.701
## - X7:X8  1   0.06075 0.37838 -85.192
## - X4:X8  1   0.06865 0.38628 -84.572
## - X2:X9  1   0.07895 0.39658 -83.783
## - X7:X9  1   0.10072 0.41835 -82.179
## - X1:X4  1   0.10847 0.42609 -81.629
## - X1:X3  1   0.52543 0.84306 -61.157
## 
## Step:  AIC=-89.89
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X2  1   0.00644 0.33001 -91.295
## <none>               0.32357 -89.886
## - X8:X9  1   0.02452 0.34809 -89.695
## - X6:X9  1   0.04082 0.36439 -88.322
## - X4:X9  1   0.04112 0.36469 -88.298
## - X1:X8  1   0.04135 0.36492 -88.278
## - X2:X8  1   0.05160 0.37517 -87.447
## - X3:X8  1   0.06741 0.39098 -86.209
## - X4:X8  1   0.07263 0.39620 -85.811
## - X2:X9  1   0.08003 0.40360 -85.256
## - X6:X8  1   0.09062 0.41419 -84.479
## - X1:X4  1   0.10555 0.42912 -83.417
## - X7:X8  1   0.13389 0.45746 -81.498
## - X7:X9  1   0.24257 0.56614 -75.103
## - X1:X3  1   0.52116 0.84473 -63.098
## 
## Step:  AIC=-91.3
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.01854 0.34855 -91.655
## <none>               0.33001 -91.295
## - X6:X9  1   0.03443 0.36444 -90.318
## - X1:X8  1   0.03853 0.36854 -89.982
## - X4:X9  1   0.04551 0.37551 -89.420
## - X2:X8  1   0.04563 0.37563 -89.410
## - X3:X8  1   0.06131 0.39132 -88.183
## - X4:X8  1   0.06990 0.39991 -87.531
## - X2:X9  1   0.07383 0.40383 -87.238
## - X6:X8  1   0.08492 0.41493 -86.425
## - X7:X8  1   0.13207 0.46208 -83.196
## - X1:X4  1   0.22142 0.55142 -77.893
## - X7:X9  1   0.25169 0.58170 -76.290
## - X1:X3  1   0.98090 1.31091 -51.914
## 
## Step:  AIC=-91.66
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.02118 0.36972 -91.886
## <none>               0.34855 -91.655
## - X1:X8  1   0.02408 0.37263 -91.651
## - X2:X8  1   0.02819 0.37674 -91.322
## - X4:X9  1   0.03669 0.38523 -90.653
## - X3:X8  1   0.04327 0.39182 -90.145
## - X4:X8  1   0.05198 0.40053 -89.485
## - X2:X9  1   0.05530 0.40385 -89.237
## - X6:X8  1   0.07115 0.41969 -88.083
## - X7:X8  1   0.11355 0.46209 -85.195
## - X1:X4  1   0.20503 0.55358 -79.777
## - X7:X9  1   0.25384 0.60238 -77.242
## - X1:X3  1   0.96420 1.31275 -53.872
## 
## Step:  AIC=-91.89
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X1:X8 + X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + X7:X8 + 
##     X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X8  1   0.00811 0.37784 -93.235
## - X2:X8  1   0.01033 0.38006 -93.059
## - X4:X9  1   0.01591 0.38563 -92.622
## <none>               0.36972 -91.886
## - X3:X8  1   0.02649 0.39621 -91.810
## - X4:X8  1   0.03080 0.40053 -91.485
## - X2:X9  1   0.03413 0.40386 -91.237
## - X6:X8  1   0.07570 0.44543 -88.298
## - X7:X8  1   0.09848 0.46820 -86.802
## - X1:X4  1   0.18408 0.55380 -81.765
## - X7:X9  1   0.29384 0.66356 -76.340
## - X1:X3  1   0.96415 1.33388 -55.393
## 
## Step:  AIC=-93.23
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X2:X8 + X2:X9 + X3:X8 + X4:X8 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X9  1   0.00950 0.38734 -94.490
## - X2:X8  1   0.01442 0.39225 -94.111
## <none>               0.37784 -93.235
## - X3:X8  1   0.03006 0.40790 -92.938
## - X2:X9  1   0.03583 0.41367 -92.517
## - X4:X8  1   0.04049 0.41833 -92.181
## - X7:X8  1   0.13216 0.50999 -86.237
## - X1:X4  1   0.20555 0.58339 -82.203
## - X7:X9  1   0.28636 0.66419 -78.311
## - X6:X8  1   0.31944 0.69728 -76.853
## - X1:X3  1   1.05532 1.43315 -55.240
## 
## Step:  AIC=-94.49
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X2:X8 + X2:X9 + X3:X8 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.00513 0.39247 -96.095
## <none>               0.38734 -94.490
## - X3:X8  1   0.02754 0.41488 -94.429
## - X2:X9  1   0.03024 0.41758 -94.234
## - X4:X8  1   0.07370 0.46104 -91.264
## - X7:X8  1   0.13414 0.52147 -87.569
## - X1:X4  1   0.20144 0.58878 -83.927
## - X7:X9  1   0.28235 0.66969 -80.064
## - X6:X8  1   0.32106 0.70840 -78.378
## - X1:X3  1   1.07234 1.45967 -56.690
## 
## Step:  AIC=-96.1
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X2:X9 + X3:X8 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X8  1   0.02353 0.41599 -96.349
## <none>               0.39247 -96.095
## - X4:X8  1   0.07216 0.46462 -93.032
## - X2:X9  1   0.13573 0.52820 -89.184
## - X7:X8  1   0.13576 0.52823 -89.183
## - X1:X4  1   0.19804 0.59051 -85.839
## - X6:X8  1   0.31907 0.71153 -80.246
## - X7:X9  1   0.33686 0.72933 -79.505
## - X1:X3  1   1.06721 1.45968 -58.689
## 
## Step:  AIC=-96.35
## log(y) ~ X1 + X2 + X3 + X4 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X2:X9 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.41599 -96.349
## - X4:X8  1   0.04956 0.46555 -94.972
## - X2:X9  1   0.11736 0.53335 -90.893
## - X1:X4  1   0.17673 0.59272 -87.727
## - X7:X8  1   0.26922 0.68521 -83.377
## - X7:X9  1   0.34394 0.75993 -80.272
## - X6:X8  1   0.36220 0.77819 -79.560
## - X1:X3  1   1.05782 1.47381 -60.400
# Interaction regression for remove X7
model_wf_rm7_log_inter <- lm(log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9)^2, data=table_wf)
model_wf_rm7_aic_log_inter <- stepAIC(model_wf_rm7_log_inter)
## Start:  AIC=-78.66
## log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9)^2
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X8 + X3:X9 + 
##     X4:X5 + X4:X6 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X8 + X3:X9 + 
##     X4:X5 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X4 + X3:X5 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X4 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X4 + X2:X8 + 
##     X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X3 + X2:X8 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + 
##     X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
## 
## Step:  AIC=-78.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.00003 0.41168 -80.661
## - X1:X9  1   0.00007 0.41173 -80.658
## - X1:X8  1   0.00009 0.41175 -80.656
## - X2:X9  1   0.00031 0.41197 -80.640
## - X6:X8  1   0.00092 0.41258 -80.596
## - X4:X9  1   0.00149 0.41315 -80.554
## - X4:X8  1   0.00252 0.41418 -80.480
## - X5:X8  1   0.00664 0.41829 -80.183
## - X5:X9  1   0.00997 0.42163 -79.945
## - X8:X9  1   0.01166 0.42331 -79.825
## - X3:X8  1   0.01730 0.42896 -79.428
## - X3:X9  1   0.01851 0.43017 -79.343
## - X1:X2  1   0.01879 0.43044 -79.324
## <none>               0.41166 -78.663
## - X1:X4  1   0.03703 0.44869 -78.079
## - X6:X9  1   0.04298 0.45463 -77.684
## - X1:X3  1   0.32537 0.73703 -63.190
## 
## Step:  AIC=-80.66
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X1:X9 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X9  1   0.00020 0.41188 -82.646
## - X1:X8  1   0.00022 0.41190 -82.645
## - X6:X8  1   0.00090 0.41258 -82.596
## - X4:X9  1   0.00148 0.41316 -82.553
## - X4:X8  1   0.00250 0.41418 -82.480
## - X8:X9  1   0.01380 0.42548 -81.672
## - X3:X8  1   0.01823 0.42991 -81.361
## - X1:X2  1   0.01882 0.43050 -81.320
## - X3:X9  1   0.02005 0.43173 -81.235
## <none>               0.41168 -80.661
## - X5:X8  1   0.03116 0.44284 -80.472
## - X1:X4  1   0.03704 0.44873 -80.076
## - X5:X9  1   0.04304 0.45472 -79.678
## - X6:X9  1   0.04316 0.45484 -79.670
## - X2:X9  1   0.05432 0.46600 -78.943
## - X1:X3  1   0.32572 0.73740 -65.175
## 
## Step:  AIC=-82.65
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X8  1   0.00002 0.41190 -84.645
## - X6:X8  1   0.00208 0.41396 -84.496
## - X4:X8  1   0.01531 0.42719 -83.551
## - X8:X9  1   0.01627 0.42815 -83.484
## - X4:X9  1   0.01882 0.43071 -83.306
## - X1:X2  1   0.02010 0.43199 -83.217
## <none>               0.41188 -82.646
## - X1:X4  1   0.03778 0.44966 -82.014
## - X5:X8  1   0.04601 0.45789 -81.470
## - X6:X9  1   0.05137 0.46325 -81.120
## - X2:X9  1   0.05779 0.46967 -80.707
## - X5:X9  1   0.06650 0.47839 -80.156
## - X3:X8  1   0.08140 0.49328 -79.236
## - X3:X9  1   0.20066 0.61254 -72.740
## - X1:X3  1   0.32682 0.73870 -67.122
## 
## Step:  AIC=-84.65
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X8  1   0.00210 0.41400 -86.492
## - X8:X9  1   0.01673 0.42862 -85.451
## - X1:X2  1   0.02018 0.43207 -85.211
## - X4:X9  1   0.02516 0.43706 -84.867
## <none>               0.41190 -84.645
## - X1:X4  1   0.03804 0.44994 -83.995
## - X5:X8  1   0.05617 0.46807 -82.810
## - X2:X9  1   0.06481 0.47671 -82.261
## - X5:X9  1   0.06698 0.47888 -82.125
## - X6:X9  1   0.06781 0.47971 -82.073
## - X4:X8  1   0.09050 0.50240 -80.687
## - X3:X8  1   0.19366 0.60556 -75.084
## - X3:X9  1   0.24559 0.65749 -72.616
## - X1:X3  1   0.32846 0.74036 -69.055
## 
## Step:  AIC=-86.49
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.01488 0.42889 -87.433
## - X1:X2  1   0.02039 0.43439 -87.050
## - X4:X9  1   0.02584 0.43984 -86.676
## <none>               0.41400 -86.492
## - X1:X4  1   0.03731 0.45131 -85.904
## - X5:X8  1   0.05665 0.47065 -84.645
## - X5:X9  1   0.06519 0.47919 -84.105
## - X2:X9  1   0.06616 0.48017 -84.045
## - X4:X8  1   0.08902 0.50302 -82.650
## - X6:X9  1   0.17440 0.58840 -77.946
## - X3:X8  1   0.19391 0.60791 -76.968
## - X3:X9  1   0.25322 0.66722 -74.175
## - X1:X3  1   0.34051 0.75451 -70.486
## 
## Step:  AIC=-87.43
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X2  1   0.00978 0.43866 -88.757
## <none>               0.42889 -87.433
## - X1:X4  1   0.03176 0.46064 -87.290
## - X5:X8  1   0.04180 0.47069 -86.643
## - X5:X9  1   0.05162 0.48051 -86.023
## - X4:X9  1   0.08907 0.51796 -83.772
## - X2:X9  1   0.11019 0.53907 -82.573
## - X4:X8  1   0.12866 0.55755 -81.562
## - X3:X8  1   0.19648 0.62536 -78.119
## - X6:X9  1   0.19882 0.62771 -78.006
## - X3:X9  1   0.24871 0.67759 -75.712
## - X1:X3  1   0.35528 0.78416 -71.330
## 
## Step:  AIC=-88.76
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X3 + X1:X4 + 
##     X2:X9 + X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X4  1   0.02659 0.46525 -88.991
## <none>               0.43866 -88.757
## - X5:X8  1   0.03564 0.47430 -88.413
## - X5:X9  1   0.04750 0.48616 -87.672
## - X4:X9  1   0.10292 0.54159 -84.433
## - X2:X9  1   0.11534 0.55400 -83.754
## - X4:X8  1   0.14344 0.58210 -82.269
## - X6:X9  1   0.19938 0.63804 -79.517
## - X3:X8  1   0.20539 0.64406 -79.235
## - X3:X9  1   0.24500 0.68366 -77.445
## - X1:X3  1   0.36833 0.80699 -72.469
## 
## Step:  AIC=-88.99
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X3 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.02961 0.49486 -89.140
## <none>               0.46525 -88.991
## - X5:X9  1   0.04496 0.51021 -88.224
## - X2:X9  1   0.10690 0.57215 -84.787
## - X4:X9  1   0.10906 0.57431 -84.673
## - X4:X8  1   0.15824 0.62350 -82.208
## - X6:X9  1   0.17629 0.64154 -81.352
## - X3:X8  1   0.18203 0.64728 -81.085
## - X3:X9  1   0.22118 0.68643 -79.323
## - X1:X3  1   2.18417 2.64942 -38.806
## 
## Step:  AIC=-89.14
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X3 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X5:X9 + X6:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X9  1   0.01576 0.51062 -90.200
## <none>               0.49486 -89.140
## - X2:X9  1   0.10256 0.59742 -85.490
## - X3:X8  1   0.15302 0.64789 -83.057
## - X6:X9  1   0.15457 0.64943 -82.986
## - X3:X9  1   0.19329 0.68815 -81.248
## - X4:X9  1   0.21327 0.70813 -80.390
## - X4:X8  1   0.39239 0.88725 -73.625
## - X1:X3  1   2.24275 2.73761 -39.823
## 
## Step:  AIC=-90.2
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X9 + X1:X3 + X2:X9 + 
##     X3:X8 + X3:X9 + X4:X8 + X4:X9 + X6:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.51062 -90.200
## - X2:X9  1   0.10538 0.61600 -86.571
## - X3:X8  1   0.14982 0.66044 -84.481
## - X3:X9  1   0.18063 0.69125 -83.114
## - X5     1   0.18248 0.69310 -83.034
## - X4:X9  1   0.20766 0.71828 -81.963
## - X6:X9  1   0.29352 0.80414 -78.575
## - X4:X8  1   0.39963 0.91025 -74.857
## - X1:X3  1   2.25494 2.76556 -41.519
# Interaction regression for remove X8
model_wf_rm8_log_inter <- lm(log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9)^2, data=table_wf)
model_wf_rm8_aic_log_inter <- stepAIC(model_wf_rm8_log_inter)
## Start:  AIC=-42.86
## log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9)^2
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X9 + X5:X6 + X5:X7 + X5:X9 + X6:X9 + 
##     X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X9 + X5:X6 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X5 + X4:X6 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X5 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X9 + 
##     X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X6 + X3:X9 + X4:X9 + 
##     X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X5 + X3:X9 + X4:X9 + X5:X9 + 
##     X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X4 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + 
##     X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X4 + X2:X9 + 
##     X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X3 + X2:X9 + X3:X9 + 
##     X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X9 + X2:X9 + X3:X9 + X4:X9 + 
##     X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X9 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + 
##     X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X9 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + 
##     X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X9 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
## 
## Step:  AIC=-42.86
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X9 + X2:X9 + X3:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X3:X9  1   0.00003 2.1656 -44.855
## - X4:X9  1   0.00459 2.1702 -44.792
## - X1:X2  1   0.01025 2.1758 -44.713
## - X5:X9  1   0.01470 2.1803 -44.652
## - X7:X9  1   0.01636 2.1819 -44.629
## - X2:X9  1   0.01823 2.1838 -44.604
## - X1:X9  1   0.02554 2.1911 -44.503
## <none>               2.1656 -42.855
## - X6:X9  1   0.26689 2.4325 -41.369
## - X1:X3  1   2.22825 4.3938 -23.630
## 
## Step:  AIC=-44.85
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X2 + X1:X3 + 
##     X1:X9 + X2:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X1:X2  1   0.01050 2.1761 -46.710
## - X2:X9  1   0.05115 2.2168 -46.154
## - X5:X9  1   0.05618 2.2218 -46.086
## - X4:X9  1   0.07602 2.2416 -45.820
## - X7:X9  1   0.12625 2.2919 -45.155
## <none>               2.1656 -44.855
## - X1:X9  1   0.35294 2.5186 -42.325
## - X6:X9  1   0.66442 2.8300 -38.827
## - X1:X3  1   2.22973 4.3953 -25.620
## 
## Step:  AIC=-46.71
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X3 + X1:X9 + 
##     X2:X9 + X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X2:X9  1   0.05946 2.2356 -47.901
## - X5:X9  1   0.06560 2.2417 -47.819
## - X4:X9  1   0.08123 2.2573 -47.610
## - X7:X9  1   0.13658 2.3127 -46.883
## <none>               2.1761 -46.710
## - X1:X9  1   0.34630 2.5224 -44.279
## - X6:X9  1   0.77425 2.9504 -39.578
## - X1:X3  1   2.26254 4.4387 -27.325
## 
## Step:  AIC=-47.9
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X3 + X1:X9 + 
##     X4:X9 + X5:X9 + X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X5:X9  1   0.05126 2.2868 -49.221
## - X4:X9  1   0.05965 2.2952 -49.111
## - X7:X9  1   0.08784 2.3234 -48.745
## <none>               2.2356 -47.901
## - X2     1   0.20548 2.4410 -47.263
## - X1:X9  1   0.39842 2.6340 -44.981
## - X6:X9  1   0.73325 2.9688 -41.391
## - X1:X3  1   2.26945 4.5050 -28.880
## 
## Step:  AIC=-49.22
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X3 + X1:X9 + 
##     X4:X9 + X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X4:X9  1   0.03562 2.3224 -50.757
## - X7:X9  1   0.15006 2.4369 -49.314
## <none>               2.2868 -49.221
## - X2     1   0.21539 2.5022 -48.521
## - X1:X9  1   0.58541 2.8722 -44.383
## - X5     1   0.64109 2.9279 -43.807
## - X6:X9  1   0.68982 2.9766 -43.312
## - X1:X3  1   2.31390 4.6007 -30.249
## 
## Step:  AIC=-50.76
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X3 + X1:X9 + 
##     X6:X9 + X7:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X7:X9  1   0.12692 2.4494 -51.161
## <none>               2.3224 -50.757
## - X2     1   0.19359 2.5160 -50.355
## - X5     1   0.60630 2.9288 -45.799
## - X6:X9  1   0.65544 2.9779 -45.299
## - X4     1   1.02227 3.3447 -41.814
## - X1:X9  1   1.07748 3.3999 -41.323
## - X1:X3  1   2.27983 4.6023 -32.239
## 
## Step:  AIC=-51.16
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X9 + X1:X3 + X1:X9 + 
##     X6:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X7     1   0.01489 2.4643 -52.979
## <none>               2.4494 -51.161
## - X2     1   0.23214 2.6815 -50.445
## - X6:X9  1   0.53611 2.9855 -47.223
## - X5     1   0.66015 3.1095 -46.002
## - X4     1   0.97813 3.4275 -43.081
## - X1:X9  1   1.69252 4.1419 -37.401
## - X1:X3  1   2.36488 4.8143 -32.888
## 
## Step:  AIC=-52.98
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X9 + X1:X3 + X1:X9 + X6:X9
## 
##         Df Sum of Sq    RSS     AIC
## <none>               2.4643 -52.979
## - X2     1    0.3580 2.8222 -50.910
## - X6:X9  1    0.5249 2.9891 -49.186
## - X5     1    0.8117 3.2760 -46.437
## - X1:X9  1    1.6800 4.1442 -39.385
## - X1:X3  1    2.9540 5.4183 -31.343
## - X4     1    3.2676 5.7319 -29.655
# Interaction regression for remove X9
model_wf_rm9_log_inter <- lm(log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8)^2, data=table_wf)
model_wf_rm9_aic_log_inter <- stepAIC(model_wf_rm9_log_inter)
## Start:  AIC=-42.8
## log(y) ~ (X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8)^2
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X8 + X5:X6 + X5:X7 + X5:X8 + X6:X8 + 
##     X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X8 + X5:X6 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X5 + X4:X6 + X4:X7 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X5 + X4:X6 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X5 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X7 + X3:X8 + 
##     X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X6 + X3:X8 + X4:X8 + 
##     X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X5 + X3:X8 + X4:X8 + X5:X8 + 
##     X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X4 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + 
##     X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X7 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X6 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X5 + 
##     X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X4 + X2:X8 + 
##     X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X3 + X2:X8 + X3:X8 + 
##     X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X7 + X1:X8 + X2:X8 + X3:X8 + X4:X8 + 
##     X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X6 + X1:X8 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + 
##     X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X5 + X1:X8 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + 
##     X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X4 + X1:X8 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
## 
## Step:  AIC=-42.8
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X8 + X2:X8 + X3:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X3:X8  1   0.00253 2.1719 -44.768
## - X2:X8  1   0.01627 2.1856 -44.579
## - X5:X8  1   0.01911 2.1885 -44.540
## - X7:X8  1   0.01945 2.1888 -44.535
## - X4:X8  1   0.02982 2.1992 -44.393
## - X1:X2  1   0.04242 2.2118 -44.222
## - X1:X8  1   0.04412 2.2135 -44.199
## <none>               2.1694 -42.803
## - X6:X8  1   0.45701 2.6264 -39.068
## - X1:X3  1   2.44526 4.6146 -22.159
## 
## Step:  AIC=-44.77
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X8 + X2:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X2:X8  1   0.01720 2.1891 -46.532
## - X5:X8  1   0.02781 2.1997 -46.386
## - X1:X2  1   0.04069 2.2126 -46.211
## - X7:X8  1   0.04552 2.2174 -46.146
## <none>               2.1719 -44.768
## - X4:X8  1   0.65876 2.8306 -38.821
## - X1:X8  1   0.78554 2.9574 -37.506
## - X6:X8  1   1.02120 3.1931 -35.206
## - X1:X3  1   2.46480 4.6367 -24.016
## 
## Step:  AIC=-46.53
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X8 + X4:X8 + X5:X8 + X6:X8 + X7:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X5:X8  1   0.02213 2.2112 -48.230
## - X7:X8  1   0.03043 2.2195 -48.117
## - X1:X2  1   0.04961 2.2387 -47.859
## <none>               2.1891 -46.532
## - X4:X8  1   0.64181 2.8309 -40.818
## - X1:X8  1   0.85315 3.0422 -38.658
## - X6:X8  1   1.01261 3.2017 -37.126
## - X1:X3  1   2.47157 4.6606 -25.861
## 
## Step:  AIC=-48.23
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X2 + X1:X3 + 
##     X1:X8 + X4:X8 + X6:X8 + X7:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X1:X2  1   0.06074 2.2719 -49.417
## - X7:X8  1   0.06248 2.2737 -49.394
## <none>               2.2112 -48.230
## - X5     1   0.54391 2.7551 -43.632
## - X4:X8  1   0.63103 2.8422 -42.698
## - X6:X8  1   1.06960 3.2808 -38.393
## - X1:X8  1   1.13203 3.3432 -37.828
## - X1:X3  1   2.52115 4.7324 -27.403
## 
## Step:  AIC=-49.42
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X3 + X1:X8 + 
##     X4:X8 + X6:X8 + X7:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X7:X8  1   0.08076 2.3527 -50.369
## <none>               2.2719 -49.417
## - X2     1   0.40486 2.6768 -46.497
## - X4:X8  1   0.57833 2.8503 -44.613
## - X5     1   0.94538 3.2173 -40.979
## - X1:X8  1   1.07683 3.3488 -39.778
## - X6:X8  1   1.11268 3.3846 -39.459
## - X1:X3  1   2.54793 4.8199 -28.853
## 
## Step:  AIC=-50.37
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X1:X3 + X1:X8 + 
##     X4:X8 + X6:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X7     1   0.01221 2.3649 -52.214
## <none>               2.3527 -50.369
## - X2     1   0.42798 2.7807 -47.355
## - X4:X8  1   0.56860 2.9213 -45.875
## - X5     1   1.01649 3.3692 -41.596
## - X1:X8  1   1.06039 3.4131 -41.207
## - X6:X8  1   1.07813 3.4308 -41.052
## - X1:X3  1   2.64896 5.0017 -29.743
## 
## Step:  AIC=-52.21
## log(y) ~ X1 + X2 + X3 + X4 + X5 + X6 + X8 + X1:X3 + X1:X8 + X4:X8 + 
##     X6:X8
## 
##         Df Sum of Sq    RSS     AIC
## <none>               2.3649 -52.214
## - X4:X8  1    0.5569 2.9218 -47.870
## - X2     1    0.5953 2.9602 -47.478
## - X1:X8  1    1.0483 3.4132 -43.206
## - X6:X8  1    1.0680 3.4329 -43.034
## - X5     1    1.1699 3.5348 -42.156
## - X1:X3  1    3.4191 5.7840 -27.383
# Interaction regression for remove X6
model_wf_rm6_log_inter <- lm(log(y) ~ (X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9)^2, data=table_wf)
model_wf_rm6_aic_log_inter <- stepAIC(model_wf_rm6_log_inter)
## Start:  AIC=-87.51
## log(y) ~ (X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X5 + X1:X4 + X1:X7 + X1:X8 + X1:X9 + 
##     X5:X4 + X5:X7 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X5 + X1:X4 + X1:X7 + X1:X8 + X1:X9 + 
##     X5:X4 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X5 + X1:X4 + X1:X7 + X1:X8 + X1:X9 + 
##     X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X5 + X1:X4 + X1:X8 + X1:X9 + X5:X8 + 
##     X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X5 + X1:X8 + X1:X9 + X5:X8 + X5:X9 + 
##     X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X7 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X8 + X5:X9 + X4:X8 + 
##     X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X4 + 
##     X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X5 + X3:X8 + 
##     X3:X9 + X1:X8 + X1:X9 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X1 + X3:X8 + X3:X9 + 
##     X1:X8 + X1:X9 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X7 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + 
##     X1:X9 + X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X4 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + 
##     X5:X8 + X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-87.51
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X8 + 
##     X5:X9 + X4:X8 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X8  1   0.00016 0.30669 -89.493
## - X5:X8  1   0.00031 0.30684 -89.479
## - X5:X9  1   0.00193 0.30846 -89.321
## - X2:X8  1   0.00428 0.31081 -89.093
## - X1:X8  1   0.00473 0.31126 -89.049
## - X3:X8  1   0.00496 0.31149 -89.028
## - X3:X9  1   0.00691 0.31345 -88.840
## - X2:X3  1   0.00840 0.31493 -88.698
## - X2:X9  1   0.01011 0.31665 -88.535
## - X4:X9  1   0.01246 0.31899 -88.314
## - X7:X8  1   0.01324 0.31978 -88.240
## <none>               0.30653 -87.509
## - X1:X9  1   0.02391 0.33044 -87.256
## - X8:X9  1   0.04485 0.35138 -85.412
## - X7:X9  1   0.13007 0.43660 -78.898
## - X2:X5  1   0.25163 0.55816 -71.529
## - X2:X1  1   0.35966 0.66620 -66.221
## 
## Step:  AIC=-89.49
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X8 + 
##     X5:X9 + X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.00024 0.30693 -91.470
## - X5:X9  1   0.00225 0.30894 -91.274
## - X2:X8  1   0.00416 0.31085 -91.090
## - X2:X3  1   0.00824 0.31493 -90.698
## - X2:X9  1   0.00996 0.31665 -90.534
## - X1:X8  1   0.01239 0.31908 -90.305
## - X7:X8  1   0.01469 0.32138 -90.090
## - X3:X8  1   0.01482 0.32151 -90.078
## <none>               0.30669 -89.493
## - X3:X9  1   0.03536 0.34205 -88.220
## - X8:X9  1   0.04571 0.35240 -87.325
## - X1:X9  1   0.05026 0.35696 -86.940
## - X7:X9  1   0.14288 0.44958 -80.019
## - X2:X5  1   0.25995 0.56664 -73.077
## - X2:X1  1   0.37446 0.68115 -67.555
## - X4:X9  1   0.37520 0.68189 -67.523
## 
## Step:  AIC=-91.47
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X3 + X2:X1 + 
##     X2:X5 + X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X9 + 
##     X4:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X3  1   0.00808 0.31502 -92.690
## - X7:X8  1   0.01895 0.32588 -91.673
## <none>               0.30693 -91.470
## - X5:X9  1   0.03833 0.34526 -89.939
## - X2:X8  1   0.04584 0.35277 -89.294
## - X3:X8  1   0.04995 0.35688 -88.946
## - X8:X9  1   0.05088 0.35781 -88.868
## - X3:X9  1   0.09199 0.39892 -85.606
## - X2:X9  1   0.09403 0.40096 -85.453
## - X1:X8  1   0.14675 0.45368 -81.747
## - X7:X9  1   0.15589 0.46282 -81.148
## - X2:X5  1   0.26313 0.57006 -74.896
## - X1:X9  1   0.27583 0.58276 -74.235
## - X2:X1  1   0.37534 0.68227 -69.506
## - X4:X9  1   0.37571 0.68264 -69.490
## 
## Step:  AIC=-92.69
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X1 + X2:X5 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X9 + X4:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X7:X8  1   0.02030 0.33531 -92.817
## <none>               0.31502 -92.690
## - X5:X9  1   0.03914 0.35416 -91.176
## - X2:X8  1   0.07963 0.39465 -87.929
## - X3:X9  1   0.08402 0.39903 -87.597
## - X8:X9  1   0.10000 0.41502 -86.419
## - X3:X8  1   0.11990 0.43491 -85.014
## - X2:X9  1   0.14379 0.45881 -83.409
## - X7:X9  1   0.15647 0.47149 -82.592
## - X1:X8  1   0.20445 0.51947 -79.684
## - X1:X9  1   0.34823 0.66325 -72.354
## - X4:X9  1   0.42560 0.74062 -69.044
## - X2:X1  1   1.07944 1.39445 -50.061
## - X2:X5  1   2.28177 2.59679 -31.408
## 
## Step:  AIC=-92.82
## log(y) ~ X2 + X3 + X1 + X5 + X4 + X7 + X8 + X9 + X2:X1 + X2:X5 + 
##     X2:X8 + X2:X9 + X3:X8 + X3:X9 + X1:X8 + X1:X9 + X5:X9 + X4:X9 + 
##     X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.33531 -92.817
## - X5:X9  1   0.04549 0.38080 -91.000
## - X3:X9  1   0.07614 0.41145 -88.678
## - X2:X8  1   0.08090 0.41621 -88.333
## - X8:X9  1   0.08897 0.42428 -87.757
## - X2:X9  1   0.14902 0.48434 -83.785
## - X3:X8  1   0.16787 0.50318 -82.640
## - X7:X9  1   0.18327 0.51859 -81.735
## - X1:X8  1   0.29429 0.62960 -75.916
## - X4:X9  1   0.42638 0.76170 -70.202
## - X1:X9  1   0.42701 0.76232 -70.178
## - X2:X1  1   1.07080 1.40612 -51.811
## - X2:X5  1   2.27861 2.61393 -33.210
# Interaction regression for remove X3
model_wf_rm3_log_inter <- lm(log(y) ~ (X1 + X2 + X4 + X5 + X6 + X7+ X8 + X9)^2, data=table_wf)
model_wf_rm3_aic_log_inter <- stepAIC(model_wf_rm3_log_inter)
## Start:  AIC=-81.16
## log(y) ~ (X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X7 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X6 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X5 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X5 + X4:X6 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X5 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X7 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X6 + 
##     X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X5 + X2:X8 + 
##     X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X4 + X2:X8 + X2:X9 + 
##     X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X4:X8 + 
##     X4:X9 + X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X6 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + 
##     X5:X8 + X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
## 
## Step:  AIC=-81.16
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X8 + X1:X9 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + 
##     X5:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X9  1   0.00269 0.38148 -82.947
## - X4:X9  1   0.00284 0.38163 -82.935
## - X6:X9  1   0.00303 0.38183 -82.919
## - X1:X8  1   0.00361 0.38240 -82.874
## - X5:X9  1   0.00401 0.38280 -82.843
## - X4:X8  1   0.00417 0.38296 -82.830
## - X8:X9  1   0.00529 0.38409 -82.743
## - X5:X8  1   0.00630 0.38509 -82.664
## - X2:X8  1   0.01232 0.39112 -82.198
## - X2:X9  1   0.01516 0.39396 -81.981
## - X6:X8  1   0.01553 0.39433 -81.953
## - X7:X8  1   0.02547 0.40426 -81.207
## <none>               0.37880 -81.159
## - X7:X9  1   0.03273 0.41152 -80.673
## - X1:X2  1   1.50396 1.88275 -35.054
## - X1:X5  1   1.52788 1.90667 -34.675
## - X1:X4  1   1.54019 1.91898 -34.482
## 
## Step:  AIC=-82.95
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X8 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.00277 0.38425 -84.730
## - X1:X8  1   0.00322 0.38470 -84.695
## - X5:X9  1   0.00360 0.38508 -84.665
## - X5:X8  1   0.00557 0.38705 -84.512
## - X4:X9  1   0.00722 0.38870 -84.384
## - X4:X8  1   0.00854 0.39002 -84.282
## - X6:X9  1   0.01101 0.39249 -84.093
## - X2:X8  1   0.01109 0.39257 -84.087
## - X2:X9  1   0.01373 0.39521 -83.886
## <none>               0.38148 -82.947
## - X6:X8  1   0.03910 0.42058 -82.019
## - X7:X8  1   0.05430 0.43579 -80.954
## - X7:X9  1   0.07530 0.45678 -79.543
## - X1:X2  1   1.50170 1.88318 -37.047
## - X1:X5  1   1.52596 1.90744 -36.663
## - X1:X4  1   1.53834 1.91983 -36.469
## 
## Step:  AIC=-84.73
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X1:X8 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + 
##     X6:X8 + X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X8  1   0.00230 0.38655 -86.551
## - X5:X9  1   0.00382 0.38807 -86.433
## - X5:X8  1   0.00521 0.38946 -86.326
## - X4:X9  1   0.00742 0.39167 -86.156
## - X4:X8  1   0.00830 0.39255 -86.089
## - X6:X9  1   0.00938 0.39363 -86.006
## - X2:X8  1   0.00976 0.39401 -85.977
## - X2:X9  1   0.01245 0.39670 -85.773
## <none>               0.38425 -84.730
## - X6:X8  1   0.03679 0.42104 -83.987
## - X7:X8  1   0.05159 0.43584 -82.950
## - X7:X9  1   0.07274 0.45699 -81.529
## - X1:X2  1   1.51822 1.90247 -38.741
## - X1:X5  1   1.53982 1.92407 -38.403
## - X1:X4  1   1.55252 1.93677 -38.205
## 
## Step:  AIC=-86.55
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X5:X9 + X6:X8 + 
##     X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X9  1   0.00600 0.39255 -88.089
## - X5:X8  1   0.00775 0.39430 -87.955
## - X4:X9  1   0.01195 0.39849 -87.638
## - X2:X8  1   0.01341 0.39996 -87.528
## - X4:X8  1   0.01483 0.40138 -87.421
## - X6:X9  1   0.01654 0.40309 -87.294
## - X2:X9  1   0.01679 0.40334 -87.275
## <none>               0.38655 -86.551
## - X6:X8  1   0.04816 0.43471 -85.028
## - X7:X8  1   0.07114 0.45768 -83.483
## - X7:X9  1   0.09037 0.47691 -82.249
## - X1:X2  1   1.55167 1.93822 -40.183
## - X1:X5  1   1.57136 1.95791 -39.880
## - X1:X4  1   1.58417 1.97071 -39.684
## 
## Step:  AIC=-88.09
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X5:X8 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X5:X8  1   0.00373 0.39628 -89.805
## - X6:X9  1   0.01256 0.40511 -89.144
## - X4:X9  1   0.01940 0.41194 -88.642
## - X2:X8  1   0.02101 0.41356 -88.524
## <none>               0.39255 -88.089
## - X4:X8  1   0.02971 0.42225 -87.900
## - X2:X9  1   0.03736 0.42991 -87.362
## - X6:X8  1   0.07926 0.47180 -84.572
## - X7:X8  1   0.15393 0.54648 -80.164
## - X7:X9  1   0.22344 0.61599 -76.572
## - X1:X2  1   1.56023 1.95278 -41.958
## - X1:X5  1   1.58079 1.97333 -41.644
## - X1:X4  1   1.59263 1.98518 -41.465
## 
## Step:  AIC=-89.81
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.00978 0.40606 -91.074
## - X4:X9  1   0.01608 0.41236 -90.612
## - X2:X8  1   0.01729 0.41356 -90.524
## <none>               0.39628 -89.805
## - X4:X8  1   0.03269 0.42897 -89.427
## - X2:X9  1   0.03398 0.43026 -89.337
## - X6:X8  1   0.07705 0.47333 -86.475
## - X7:X8  1   0.15021 0.54649 -82.163
## - X7:X9  1   0.22138 0.61766 -78.490
## - X1:X2  1   1.56477 1.96105 -43.832
## - X1:X5  1   1.58531 1.98159 -43.519
## - X1:X4  1   1.59714 1.99342 -43.340
## 
## Step:  AIC=-91.07
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X8 + X2:X9 + X4:X8 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X2:X8  1   0.00817 0.41423 -92.476
## - X4:X9  1   0.00873 0.41479 -92.435
## - X4:X8  1   0.02404 0.43010 -91.348
## - X2:X9  1   0.02424 0.43030 -91.334
## <none>               0.40606 -91.074
## - X7:X8  1   0.19396 0.60001 -81.360
## - X7:X9  1   0.26498 0.67104 -78.004
## - X6:X8  1   0.34931 0.75537 -74.452
## - X1:X2  1   1.63453 2.04059 -44.639
## - X1:X5  1   1.64952 2.05558 -44.419
## - X1:X4  1   1.66294 2.06900 -44.224
## 
## Step:  AIC=-92.48
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X9 + X4:X8 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X9  1   0.00158 0.41581 -94.362
## - X4:X8  1   0.01848 0.43271 -93.166
## <none>               0.41423 -92.476
## - X2:X9  1   0.11733 0.53156 -86.994
## - X7:X8  1   0.18863 0.60286 -83.218
## - X7:X9  1   0.26669 0.68092 -79.565
## - X6:X8  1   0.34144 0.75567 -76.441
## - X1:X2  1   1.70332 2.11755 -45.528
## - X1:X5  1   1.71559 2.12982 -45.355
## - X1:X4  1   1.72979 2.14402 -45.155
## 
## Step:  AIC=-94.36
## log(y) ~ X1 + X2 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X2 + X1:X4 + 
##     X1:X5 + X2:X9 + X4:X8 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.41581 -94.362
## - X4:X8  1   0.04971 0.46552 -92.974
## - X2:X9  1   0.11751 0.53332 -88.895
## - X7:X8  1   0.26340 0.67921 -81.641
## - X7:X9  1   0.34302 0.75883 -78.315
## - X6:X8  1   0.36050 0.77631 -77.632
## - X1:X2  1   1.70614 2.12194 -47.466
## - X1:X5  1   1.71693 2.13274 -47.314
## - X1:X4  1   1.73128 2.14708 -47.113
# Interaction regression for 136789
model_wf_136789_log_inter <- lm(log(y) ~ (X3 + X1 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_136789_aic_log_inter <- stepAIC(model_wf_136789_log_inter)
## Start:  AIC=-80.83
## log(y) ~ (X3 + X1 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-80.83
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X3:X8 + X3:X9 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.00038 0.50037 -82.808
## - X3:X8  1   0.00100 0.50099 -82.771
## - X8:X9  1   0.00137 0.50136 -82.749
## - X1:X8  1   0.00290 0.50289 -82.657
## - X6:X9  1   0.00372 0.50371 -82.608
## - X1:X9  1   0.01443 0.51442 -81.977
## - X1:X6  1   0.02812 0.52811 -81.189
## <none>               0.49999 -80.831
## - X6:X8  1   0.04039 0.54038 -80.500
## - X7:X9  1   0.05975 0.55974 -79.444
## - X7:X8  1   0.06080 0.56079 -79.388
## - X3:X6  1   0.14820 0.64819 -75.043
## - X3:X1  1   0.21507 0.71506 -72.098
## - X1:X7  1   0.29833 0.79832 -68.793
## - X3:X7  1   0.51268 1.01267 -61.658
## 
## Step:  AIC=-82.81
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X3:X8 + X1:X6 + X1:X7 + X1:X8 + X1:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X8  1   0.00191 0.50228 -84.694
## - X8:X9  1   0.00251 0.50288 -84.658
## - X1:X8  1   0.00367 0.50404 -84.589
## - X6:X9  1   0.00571 0.50608 -84.468
## - X1:X9  1   0.02266 0.52303 -83.479
## - X1:X6  1   0.02833 0.52870 -83.156
## <none>               0.50037 -82.808
## - X6:X8  1   0.09020 0.59057 -79.836
## - X7:X8  1   0.13097 0.63134 -77.833
## - X7:X9  1   0.14578 0.64615 -77.138
## - X3:X6  1   0.14928 0.64965 -76.976
## - X3:X1  1   0.21921 0.71958 -73.908
## - X1:X7  1   0.30938 0.80974 -70.367
## - X3:X7  1   0.52231 1.02268 -63.363
## 
## Step:  AIC=-84.69
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X1:X6 + X1:X7 + X1:X8 + X1:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.00269 0.50496 -86.534
## - X1:X8  1   0.00444 0.50671 -86.430
## - X6:X9  1   0.00606 0.50834 -86.334
## - X1:X9  1   0.02711 0.52939 -85.117
## - X1:X6  1   0.02748 0.52975 -85.096
## <none>               0.50228 -84.694
## - X6:X8  1   0.09254 0.59482 -81.621
## - X7:X8  1   0.13034 0.63262 -79.772
## - X7:X9  1   0.15587 0.65815 -78.586
## - X3:X6  1   0.18423 0.68650 -77.320
## - X3:X1  1   0.22216 0.72444 -75.707
## - X1:X7  1   0.31471 0.81698 -72.100
## - X3:X7  1   0.52929 1.03157 -65.103
## 
## Step:  AIC=-86.53
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X1:X6 + X1:X7 + X1:X8 + X1:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X1:X8  1   0.00271 0.50767 -88.374
## - X6:X9  1   0.00509 0.51005 -88.233
## - X1:X9  1   0.02463 0.52959 -87.105
## - X1:X6  1   0.02868 0.53364 -86.877
## <none>               0.50496 -86.534
## - X6:X8  1   0.08986 0.59482 -83.621
## - X7:X8  1   0.13060 0.63556 -81.633
## - X7:X9  1   0.15845 0.66341 -80.347
## - X3:X6  1   0.18168 0.68664 -79.314
## - X3:X1  1   0.22010 0.72506 -77.681
## - X1:X7  1   0.31247 0.81743 -74.084
## - X3:X7  1   0.52949 1.03445 -67.020
## 
## Step:  AIC=-88.37
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X1:X6 + X1:X7 + X1:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.00296 0.51064 -90.199
## - X1:X6  1   0.03465 0.54232 -88.393
## <none>               0.50767 -88.374
## - X6:X8  1   0.10273 0.61041 -84.845
## - X1:X9  1   0.10325 0.61092 -84.820
## - X3:X6  1   0.17906 0.68674 -81.310
## - X3:X1  1   0.23140 0.73908 -79.107
## - X7:X8  1   0.23214 0.73981 -79.077
## - X1:X7  1   0.33295 0.84062 -75.244
## - X7:X9  1   0.33515 0.84282 -75.166
## - X3:X7  1   0.56759 1.07526 -67.859
## 
## Step:  AIC=-90.2
## log(y) ~ X3 + X1 + X6 + X7 + X8 + X9 + X3:X1 + X3:X6 + X3:X7 + 
##     X1:X6 + X1:X7 + X1:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.51064 -90.199
## - X1:X6  1   0.03796 0.54860 -90.048
## - X1:X9  1   0.14195 0.65259 -84.840
## - X3:X6  1   0.17884 0.68948 -83.191
## - X3:X1  1   0.22900 0.73964 -81.084
## - X7:X8  1   0.29375 0.80439 -78.566
## - X1:X7  1   0.33096 0.84159 -77.210
## - X7:X9  1   0.36898 0.87962 -75.884
## - X6:X8  1   0.47374 0.98438 -72.508
## - X3:X7  1   0.57859 1.08923 -69.472
# Interaction regression for 436789
model_wf_436789_log_inter <- lm(log(y) ~ (X3 + X4 + X6 + X7 + X8 + X9)^2, data=table_wf)
model_wf_436789_aic_log_inter <- stepAIC(model_wf_436789_log_inter)
## Start:  AIC=-80.59
## log(y) ~ (X3 + X4 + X6 + X7 + X8 + X9)^2
## 
## 
## Step:  AIC=-80.59
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X3:X8 + X3:X9 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + 
##     X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X8  1   0.00049 0.50455 -82.558
## - X4:X8  1   0.00061 0.50468 -82.551
## - X3:X9  1   0.00397 0.50804 -82.352
## - X6:X9  1   0.00772 0.51179 -82.131
## - X8:X9  1   0.01022 0.51428 -81.985
## - X4:X9  1   0.01257 0.51664 -81.848
## - X4:X6  1   0.02379 0.52785 -81.204
## <none>               0.50407 -80.587
## - X6:X8  1   0.04716 0.55123 -79.904
## - X3:X4  1   0.07218 0.57625 -78.572
## - X7:X8  1   0.07701 0.58108 -78.322
## - X7:X9  1   0.08040 0.58447 -78.147
## - X3:X6  1   0.13329 0.63735 -75.549
## - X4:X7  1   0.20436 0.70843 -72.377
## - X3:X7  1   0.39949 0.90355 -65.079
## 
## Step:  AIC=-82.56
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X3:X9 + X4:X6 + X4:X7 + X4:X8 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + 
##     X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X8  1   0.00013 0.50468 -84.551
## - X8:X9  1   0.01122 0.51578 -83.898
## - X6:X9  1   0.01243 0.51698 -83.828
## - X3:X9  1   0.01559 0.52014 -83.646
## - X4:X9  1   0.02032 0.52488 -83.374
## - X4:X6  1   0.02341 0.52796 -83.198
## <none>               0.50455 -82.558
## - X3:X4  1   0.07253 0.57709 -80.529
## - X6:X8  1   0.09561 0.60016 -79.353
## - X3:X6  1   0.13301 0.63756 -77.539
## - X7:X9  1   0.20108 0.70563 -74.496
## - X7:X8  1   0.20133 0.70588 -74.485
## - X4:X7  1   0.20989 0.71445 -74.123
## - X3:X7  1   0.39994 0.90449 -67.047
## 
## Step:  AIC=-84.55
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X3:X9 + X4:X6 + X4:X7 + X4:X9 + X6:X8 + X6:X9 + X7:X8 + X7:X9 + 
##     X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X6:X9  1   0.01278 0.51747 -85.800
## - X8:X9  1   0.01313 0.51781 -85.780
## - X3:X9  1   0.01876 0.52344 -85.456
## - X4:X6  1   0.02633 0.53101 -85.025
## <none>               0.50468 -84.551
## - X3:X4  1   0.07754 0.58223 -82.263
## - X4:X9  1   0.08811 0.59280 -81.723
## - X6:X8  1   0.10413 0.60881 -80.923
## - X3:X6  1   0.16091 0.66559 -78.248
## - X4:X7  1   0.23295 0.73763 -75.165
## - X7:X8  1   0.34126 0.84594 -71.055
## - X7:X9  1   0.40457 0.90925 -68.890
## - X3:X7  1   0.42152 0.92621 -68.336
## 
## Step:  AIC=-85.8
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X3:X9 + X4:X6 + X4:X7 + X4:X9 + X6:X8 + X7:X8 + X7:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.01467 0.53213 -86.962
## - X3:X9  1   0.01520 0.53267 -86.932
## - X4:X6  1   0.03010 0.54756 -86.104
## <none>               0.51747 -85.800
## - X3:X4  1   0.07026 0.58773 -83.981
## - X4:X9  1   0.09708 0.61454 -82.642
## - X3:X6  1   0.14976 0.66722 -80.175
## - X4:X7  1   0.22310 0.74056 -77.046
## - X6:X8  1   0.29414 0.81161 -74.298
## - X7:X8  1   0.35360 0.87107 -72.177
## - X3:X7  1   0.41037 0.92784 -70.283
## - X7:X9  1   0.41188 0.92934 -70.234
## 
## Step:  AIC=-86.96
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X3:X9 + X4:X6 + X4:X7 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X3:X9  1   0.01166 0.54379 -88.311
## - X4:X6  1   0.03147 0.56360 -87.238
## <none>               0.53213 -86.962
## - X3:X4  1   0.06446 0.59659 -85.532
## - X4:X9  1   0.08245 0.61458 -84.640
## - X3:X6  1   0.14800 0.68013 -81.600
## - X4:X7  1   0.21743 0.74956 -78.684
## - X6:X8  1   0.31543 0.84756 -74.998
## - X7:X8  1   0.35376 0.88589 -73.671
## - X3:X7  1   0.39957 0.93170 -72.158
## - X7:X9  1   0.40003 0.93216 -72.144
## 
## Step:  AIC=-88.31
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X4:X6 + X4:X7 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X6  1   0.02894 0.57273 -88.756
## <none>               0.54379 -88.311
## - X3:X4  1   0.07020 0.61399 -86.669
## - X4:X9  1   0.10879 0.65259 -84.840
## - X3:X6  1   0.13747 0.68126 -83.550
## - X4:X7  1   0.22959 0.77338 -79.745
## - X6:X8  1   0.37013 0.91393 -74.736
## - X7:X9  1   0.39405 0.93784 -73.961
## - X3:X7  1   0.42811 0.97191 -72.891
## - X7:X8  1   0.46631 1.01010 -71.734
## 
## Step:  AIC=-88.76
## log(y) ~ X3 + X4 + X6 + X7 + X8 + X9 + X3:X4 + X3:X6 + X3:X7 + 
##     X4:X7 + X4:X9 + X6:X8 + X7:X8 + X7:X9
## 
##         Df Sum of Sq     RSS     AIC
## <none>               0.57273 -88.756
## - X3:X6  1   0.11387 0.68660 -85.316
## - X3:X4  1   0.11610 0.68883 -85.219
## - X4:X9  1   0.12843 0.70116 -84.687
## - X6:X8  1   0.34761 0.92034 -76.526
## - X7:X9  1   0.38814 0.96087 -75.233
## - X7:X8  1   0.45787 1.03060 -73.132
## - X3:X7  1   0.51773 1.09046 -71.438
## - X4:X7  1   0.51883 1.09157 -71.408
# Interaction regression for 437
model_wf_437_log_inter <- lm(log(y) ~ (X3 + X4 + X7 )^2, data=table_wf)
model_wf_437_aic_log_inter <- stepAIC(model_wf_437_log_inter)
## Start:  AIC=-36.51
## log(y) ~ (X3 + X4 + X7)^2
## 
##         Df Sum of Sq    RSS     AIC
## - X3:X4  1   0.06155 5.6315 -38.185
## <none>               5.5699 -36.514
## - X3:X7  1   0.75195 6.3219 -34.715
## - X4:X7  1   0.80040 6.3703 -34.486
## 
## Step:  AIC=-38.18
## log(y) ~ X3 + X4 + X7 + X3:X7 + X4:X7
## 
##         Df Sum of Sq    RSS     AIC
## <none>               5.6315 -38.185
## - X3:X7  1    1.1098 6.7412 -34.789
## - X4:X7  1    1.5395 7.1710 -32.935
# Interaction regression for 489
model_wf_489_log_inter <- lm(log(y) ~ (X4 + X8 + X9 )^2, data=table_wf)
model_wf_489_aic_log_inter <- stepAIC(model_wf_489_log_inter)
## Start:  AIC=-24.34
## log(y) ~ (X4 + X8 + X9)^2
## 
##         Df Sum of Sq    RSS     AIC
## - X4:X9  1   0.50244 8.8596 -24.591
## - X8:X9  1   0.56225 8.9194 -24.389
## <none>               8.3572 -24.342
## - X4:X8  1   0.74429 9.1015 -23.783
## 
## Step:  AIC=-24.59
## log(y) ~ X4 + X8 + X9 + X4:X8 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X4:X8  1   0.26521 9.1248 -25.706
## - X8:X9  1   0.28528 9.1449 -25.640
## <none>               8.8596 -24.591
## 
## Step:  AIC=-25.71
## log(y) ~ X4 + X8 + X9 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## <none>                9.125 -25.706
## - X8:X9  1     0.737  9.862 -25.375
## - X4     1    50.366 59.491  28.539
# Interaction regression by groups
model_wf_3g_log_inter <- lm(log(y) ~ (X1 + X3 + X4 )^2 + (X2+ X5+X6 + X7)^2 + (X8 + X9)^2, data=table_wf)
model_wf_3g_aic_log_inter <- stepAIC(model_wf_3g_log_inter)
## Start:  AIC=-60.2
## log(y) ~ (X1 + X3 + X4)^2 + (X2 + X5 + X6 + X7)^2 + (X8 + X9)^2
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X2:X5 + X2:X6 + X2:X7 + X5:X6 + X5:X7 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X2:X5 + X2:X6 + X2:X7 + X5:X6 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X2:X5 + X2:X6 + X2:X7 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X2:X5 + X2:X6 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X2:X5 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X3:X4 + X8:X9
## 
## 
## Step:  AIC=-60.2
## log(y) ~ X1 + X3 + X4 + X2 + X5 + X6 + X7 + X8 + X9 + X1:X3 + 
##     X1:X4 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X2     1  0.000002 1.6952 -62.202
## - X6     1  0.000166 1.6953 -62.200
## - X1:X4  1  0.000275 1.6954 -62.198
## - X5     1  0.000945 1.6961 -62.186
## - X7     1  0.001015 1.6962 -62.185
## - X1:X3  1  0.026614 1.7218 -61.735
## <none>               1.6952 -60.203
## - X8:X9  1  0.138718 1.8339 -59.843
## 
## Step:  AIC=-62.2
## log(y) ~ X1 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X1:X3 + X1:X4 + 
##     X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X6     1   0.00623 1.7014 -64.092
## - X7     1   0.02474 1.7199 -63.768
## - X5     1   0.07614 1.7713 -62.884
## <none>               1.6952 -62.202
## - X8:X9  1   0.13956 1.8347 -61.829
## - X1:X4  1   0.23565 1.9308 -60.298
## - X1:X3  1   1.83721 3.5324 -42.177
## 
## Step:  AIC=-64.09
## log(y) ~ X1 + X3 + X4 + X5 + X7 + X8 + X9 + X1:X3 + X1:X4 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X7     1   0.02011 1.7215 -65.740
## - X5     1   0.07456 1.7760 -64.806
## <none>               1.7014 -64.092
## - X8:X9  1   0.14360 1.8450 -63.662
## - X1:X4  1   0.23511 1.9365 -62.209
## - X1:X3  1   2.31225 4.0137 -40.345
## 
## Step:  AIC=-65.74
## log(y) ~ X1 + X3 + X4 + X5 + X8 + X9 + X1:X3 + X1:X4 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X5     1   0.10799 1.8295 -65.915
## <none>               1.7215 -65.740
## - X8:X9  1   0.13002 1.8515 -65.556
## - X1:X4  1   0.36723 2.0887 -61.939
## - X1:X3  1   2.30410 4.0256 -42.256
## 
## Step:  AIC=-65.91
## log(y) ~ X1 + X3 + X4 + X8 + X9 + X1:X3 + X1:X4 + X8:X9
## 
##         Df Sum of Sq    RSS     AIC
## - X8:X9  1   0.12305 1.9525 -65.962
## <none>               1.8295 -65.915
## - X1:X4  1   0.26634 2.0958 -63.837
## - X1:X3  1   2.46788 4.2974 -42.296
## 
## Step:  AIC=-65.96
## log(y) ~ X1 + X3 + X4 + X8 + X9 + X1:X3 + X1:X4
## 
##         Df Sum of Sq    RSS     AIC
## <none>               1.9525 -65.962
## - X1:X4  1    0.2868 2.2393 -63.851
## - X1:X3  1    2.4981 4.4506 -43.245
## - X9     1    3.0987 5.0513 -39.447
## - X8     1    3.5877 5.5402 -36.675
# Interaction regression by groups1
model_wf_3g1_log_inter <- lm(log(y) ~ (X1 + X2 + X5 )^2 + (X3 +X6 + X7)^2 + (X4 +X8 + X9)^2, data=table_wf)
model_wf_3g1_aic_log_inter <- stepAIC(model_wf_3g1_log_inter)
## Start:  AIC=-74.84
## log(y) ~ (X1 + X2 + X5)^2 + (X3 + X6 + X7)^2 + (X4 + X8 + X9)^2
## 
## 
## Step:  AIC=-74.84
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X2:X5 + X3:X6 + X3:X7 + X4:X8 + X4:X9 + X8:X9
## 
## 
## Step:  AIC=-74.84
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X2:X5 + X3:X6 + X4:X8 + X4:X9 + X8:X9
## 
## 
## Step:  AIC=-74.84
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X2:X5 + X4:X8 + X4:X9 + X8:X9
## 
## 
## Step:  AIC=-74.84
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X4:X8 + X4:X9 + X8:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X8:X9  1   0.00073 0.91149 -76.816
## - X4:X9  1   0.04009 0.95085 -75.548
## <none>               0.91076 -74.840
## - X1:X2  1   0.07105 0.98182 -74.586
## - X7     1   0.17174 1.08250 -71.658
## - X4:X8  1   0.26611 1.17687 -69.150
## - X6     1   0.55909 1.46985 -62.481
## - X3     1   1.76408 2.67484 -44.519
## - X1:X5  1   1.76865 2.67941 -44.468
## 
## Step:  AIC=-76.82
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X4:X8 + X4:X9
## 
##         Df Sum of Sq     RSS     AIC
## - X4:X9  1   0.04513 0.95662 -77.366
## <none>               0.91149 -76.816
## - X1:X2  1   0.07035 0.98184 -76.586
## - X7     1   0.17145 1.08294 -73.645
## - X4:X8  1   0.27962 1.19111 -70.789
## - X6     1   0.57239 1.48388 -64.196
## - X3     1   1.76478 2.67627 -46.503
## - X1:X5  1   1.76792 2.67941 -46.468
## 
## Step:  AIC=-77.37
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X2 + 
##     X1:X5 + X4:X8
## 
##         Df Sum of Sq    RSS     AIC
## - X1:X2  1   0.04916 1.0058 -77.863
## <none>               0.9566 -77.366
## - X7     1   0.14180 1.0984 -75.220
## - X6     1   0.52740 1.4840 -66.193
## - X4:X8  1   0.87727 1.8339 -59.843
## - X3     1   1.74611 2.7027 -48.208
## - X1:X5  1   1.78951 2.7461 -47.730
## - X9     1   3.09207 4.0487 -36.084
## 
## Step:  AIC=-77.86
## log(y) ~ X1 + X2 + X5 + X3 + X6 + X7 + X4 + X8 + X9 + X1:X5 + 
##     X4:X8
## 
##         Df Sum of Sq    RSS     AIC
## <none>               1.0058 -77.863
## - X7     1   0.19415 1.1999 -74.568
## - X6     1   0.60076 1.6065 -65.813
## - X4:X8  1   0.91902 1.9248 -60.391
## - X2     1   1.15826 2.1640 -56.877
## - X3     1   1.77019 2.7760 -49.406
## - X1:X5  1   1.77427 2.7800 -49.362
## - X9     1   3.07618 4.0820 -37.839
# Comparison
huxreg(model_wf_rm1_aic_log_inter, model_wf_rm2_aic_log_inter, model_wf_rm3_aic_log_inter, model_wf_rm4_aic_log_inter, model_wf_rm5_aic_log_inter, model_wf_rm6_aic_log_inter, model_wf_rm7_aic_log_inter, model_wf_rm8_aic_log_inter, model_wf_rm9_aic_log_inter, model_wf_aic_log_inter)
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10)
(Intercept) 12.952 **  1.257     -1168.842 *** -0.921     2.113 *** 20.435 **  -0.437     -3.177     -5.990 *   -0.981    
(3.352)    (4.109)    (160.410)    (1.353)    (0.465)    (5.318)    (1.533)    (2.424)    (2.579)    (1.520)   
X2 -557.336 ***          -12136.443 *** -29.959 *** 15.539     -1331.530 *** -16.910 **  -9.262     -12.071 *   -24.452 ***
(94.294)             (1652.799)    (5.153)    (12.186)    (169.898)    (5.434)    (5.575)    (5.671)    (5.631)   
X3 0.334 *** 0.319 *            0.409 *** 0.325 *** 0.362 *   0.368 *** 0.457 *** 0.505 *** 0.493 ***
(0.069)    (0.129)             (0.047)    (0.042)    (0.111)    (0.062)    (0.091)    (0.093)    (0.066)   
X4 0.105 *   0.479 *   4.739 ***          0.694 *** 0.073     0.306 *** 0.279 *** 0.404 *** 0.168 ** 
(0.036)    (0.161)    (0.645)             (0.135)    (0.116)    (0.038)    (0.055)    (0.087)    (0.054)   
X5 -0.137 **  0.017     17.145 *** 0.047 **           -0.253 **  0.039 *   0.072 *   0.090 **  0.039 *  
(0.045)    (0.049)    (2.344)    (0.015)             (0.067)    (0.017)    (0.029)    (0.030)    (0.016)   
X6 0.533 *   0.792     -26.739 *** 0.406 *   0.792 **           0.376 *   0.544     0.907 *           
(0.197)    (0.392)    (3.702)    (0.171)    (0.218)             (0.163)    (0.266)    (0.317)            
X7 -3.841 *   -3.083     257.995 *** 0.356     -3.131     -0.444                                2.515    
(1.690)    (2.708)    (35.274)    (1.413)    (1.593)    (2.305)                               (1.258)   
X8 0.118     2.373     0.118     0.173     0.115     1.004 **  0.574 ***          0.510 *   0.586 ***
(0.183)    (2.677)    (0.183)    (0.196)    (0.173)    (0.273)    (0.134)             (0.192)    (0.087)   
X9 -0.021     -2.356     -0.021     -0.016     -0.020     0.979     -0.454 **  0.021              -0.248 *  
(0.158)    (2.129)    (0.158)    (0.150)    (0.153)    (1.151)    (0.115)    (0.124)             (0.105)   
X2:X3 -12.335 ***                                                                                 
(2.590)                                                                                    
X2:X4 8.606 ***                                                                                 
(1.788)                                                                                    
X2:X5 8.144 ***                                     18.241 ***                                    
(1.315)                                        (2.332)                                       
X2:X9 2.574              2.574     2.513     2.568     20.237     2.238                       4.643 ** 
(1.343)             (1.343)    (1.280)    (1.292)    (10.119)    (1.316)                      (1.449)   
X4:X8 0.009     0.166     0.009              0.009              0.052 **           -0.043     0.071 ***
(0.007)    (0.212)    (0.007)             (0.007)             (0.016)             (0.021)    (0.015)   
X6:X8 -0.178 **  -0.232     -0.178 **  -0.189 **  -0.178 **                             -0.281 *           
(0.053)    (0.117)    (0.053)    (0.050)    (0.051)                               (0.099)            
X7:X8 2.215 *   2.057     2.215 *   2.046 *   2.223 **                                              
(0.772)    (1.528)    (0.772)    (0.816)    (0.738)                                                
X7:X9 -1.934 **  -1.397     -1.934 **  -1.844 **  -1.935 **  -1.514                                -0.995 *  
(0.590)    (1.231)    (0.590)    (0.591)    (0.569)    (0.683)                               (0.344)   
X1          1.638 **  1398.510 *** -0.982     -0.025     -0.165     2.278 *** 2.479 *** 2.569 *** 2.712 ***
         (0.476)    (190.396)    (0.516)    (0.829)    (0.402)    (0.350)    (0.630)    (0.633)    (0.330)   
X1:X3          -0.484 ***          -0.217 *** -0.599 ***          -0.399 *** -0.437 *** -0.478 *** -0.416 ***
         (0.085)             (0.044)    (0.100)             (0.051)    (0.092)    (0.094)    (0.045)   
X1:X4          0.035     -37.499 ***          0.126 *                                               
         (0.017)    (5.097)             (0.052)                                                
X1:X8          -0.444              0.025              0.317 *                     0.143 *           
         (0.552)             (0.018)             (0.113)                      (0.051)            
X1:X9          0.407                                -0.450 **           0.062 **           -0.118 ** 
         (0.428)                               (0.133)             (0.017)             (0.036)   
X3:X8          -0.109                                -0.046     -0.045                       -0.051 ** 
         (0.142)                               (0.022)    (0.022)                      (0.014)   
X3:X9          0.101                                -0.043     0.039 *                             
         (0.110)                               (0.030)    (0.018)                              
X4:X9          -0.137                                0.091 **  -0.038 *                             
         (0.160)                               (0.027)    (0.016)                              
X5:X8          -0.026                                                                            
         (0.031)                                                                           
X5:X9          0.027                                -0.015                                        
         (0.024)                               (0.014)                                       
X1:X2                   1244.211 ***                                                               
                  (170.358)                                                                  
X1:X5                   -11.640 *** 0.052 ***                                                      
                  (1.589)    (0.007)                                                         
X2:X1                                              40.080 ***                                    
                                             (7.476)                                       
X2:X8                                              -19.045                                        
                                             (12.925)                                       
X8:X9                                              -0.051                                        
                                             (0.033)                                       
X6:X9                                                       -0.124 *   -0.150                      
                                                      (0.044)    (0.074)                     
N 30         30         30         30         30         30         30         30         30         30        
R2 0.994     0.994     0.994     0.994     0.994     0.995     0.993     0.965     0.967     0.994    
logLik 21.613     20.677     21.613     21.546     21.606     24.840     18.532     -5.079     -4.461     20.797    
AIC -7.226     4.647     -7.226     -9.091     -9.212     -5.680     -3.063     34.157     34.923     -9.594    
*** p < 0.001; ** p < 0.01; * p < 0.05.
huxreg(model_wf_136789_aic_log_inter,model_wf_436789_aic_log_inter, model_wf_437_log_inter, model_wf_489_log_inter, model_wf_3g_aic_log_inter, model_wf_3g1_aic_log_inter)
(1) (2) (3) (4) (5) (6)
(Intercept) 14.879 *** 10.906 *** 8.147 * 5.177 *** 2.409 *** 1.057    
(3.065)    (2.124)    (3.182)  (0.641)    (0.302)    (1.829)   
X3 -1.479 **  -0.673 *   -0.318            0.304 *** 0.427 ***
(0.418)    (0.259)    (0.402)           (0.052)    (0.076)   
X1 -4.829 *                            1.336 *** -3.625 ***
(1.686)                             (0.305)    (0.655)   
X6 -0.078     0.210                              -0.386 ** 
(0.421)    (0.309)                             (0.118)   
X7 -47.166 **  -31.449 **  -20.714                     2.442    
(13.800)    (8.828)    (12.111)                    (1.310)   
X8 0.151     0.060            0.106     0.612 *** 0.402 ***
(0.202)    (0.191)           (0.335)    (0.096)    (0.092)   
X9 0.005     -0.059            -0.599 *   -0.471 *** -0.509 ***
(0.156)    (0.164)           (0.277)    (0.080)    (0.069)   
X3:X1 0.321 *                                             
(0.128)                                              
X3:X6 0.064 *   0.043                                      
(0.029)    (0.025)                                     
X3:X7 7.144 **  4.043 **  2.839                             
(1.794)    (1.098)    (1.611)                            
X1:X6 0.043                                               
(0.042)                                              
X1:X7 5.824 **                                            
(1.933)                                              
X1:X9 0.037                                               
(0.019)                                              
X6:X8 -0.200 **  -0.169 **                                   
(0.056)    (0.056)                                     
X7:X8 2.208 *   2.536 **                                   
(0.778)    (0.732)                                     
X7:X9 -1.804 **  -1.843 **                                   
(0.567)    (0.578)                                     
X4          -0.699 *   -0.439   0.184 *** 0.401 *** -0.403 ** 
         (0.256)    (0.399)  (0.048)    (0.056)    (0.103)   
X3:X4          0.024     0.011                             
         (0.014)    (0.023)                            
X4:X7          1.357 **  1.021                             
         (0.368)    (0.562)                            
X4:X9          0.011            -0.040                      
         (0.006)           (0.034)                     
X4:X8                          0.050              0.022 ***
                         (0.035)             (0.005)   
X8:X9                          0.088                      
                         (0.070)                     
X1:X3                                   -0.390 ***         
                                  (0.074)            
X1:X4                                   0.029             
                                  (0.016)            
X2                                            -29.815 ***
                                           (6.549)   
X5                                            0.035    
                                           (0.022)   
X1:X5                                            0.090 ***
                                           (0.016)   
N 30         30         30       30         30         30        
R2 0.993     0.992     0.922   0.883     0.973     0.986    
logLik 18.531     16.810     -17.311   -23.397     -1.587     8.363    
AIC -3.063     -1.620     50.622   62.794     21.174     9.273    
*** p < 0.001; ** p < 0.01; * p < 0.05.
  • Other Models
# build X1*X8 eliminated log model
model_wf_18rm4_log <- lm(log(y) ~ X1*X8 + X3 + X6 + X7 + X9, data=table_wf)

# build X1*X8 eliminated log model
table_wf_resi <- table_wf%>% mutate(x1t8=X1*X8)
model_wf_1time8_log <- lm(log(y) ~ x1t8 + X3 + X6 + X7+ X9 , data=table_wf_resi)

# build X1*X4 eliminated log model
table_wf_resi <- table_wf%>% mutate(x1t4=X1*X4)
model_wf_1time4_log <- lm(log(y) ~ x1t4 + X3 + X6 + X7+ X8+ X9, data=table_wf_resi)
summary(model_wf_1time4_log)
## 
## Call:
## lm(formula = log(y) ~ x1t4 + X3 + X6 + X7 + X8 + X9, data = table_wf_resi)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.0156 -0.2543  0.0070  0.2564  0.6057 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  2.139295   0.393809   5.432 1.61e-05 ***
## x1t4         0.009493   0.002504   3.791 0.000945 ***
## X3           0.274661   0.021379  12.847 5.60e-12 ***
## X6          -0.557992   0.145847  -3.826 0.000866 ***
## X7           6.105175   0.885304   6.896 4.96e-07 ***
## X8           0.615842   0.138876   4.434 0.000191 ***
## X9          -0.435302   0.113260  -3.843 0.000829 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.4259 on 23 degrees of freedom
## Multiple R-squared:  0.9416, Adjusted R-squared:  0.9263 
## F-statistic: 61.76 on 6 and 23 DF,  p-value: 4.956e-13
# build X1/X4 eliminated log model
table_wf_resi <- table_wf%>% mutate(x14=X1/X4)
model_wf_1per4_log <- lm(log(y) ~ x14 + X3 + X6 + X7+ X8+ X9, data=table_wf_resi)

# build X4*X3 eliminated log model
model_wf_43rm1_log <- lm(log(y) ~ X9 + X4*X3 + X6 + X7 + X8 , data=table_wf)

# build X4*X9 eliminated log model
model_wf_49rm1_log <- lm(log(y) ~ X3 + X4*X9 + X6 + X7 + X8 , data=table_wf)

# build X4*X9 eliminated log model
model_wf_48rm1_log <- lm(log(y) ~ X3 + X4*X8 + X6 + X7 + X9 , data=table_wf)

# build X4*X9 eliminated log model
model_wf_47rm1_log <- lm(log(y) ~ X3 + X4*X7 + X6 + X9 + X8 , data=table_wf)

# build X4/X9 eliminated log model
table_wf_resi <- table_wf%>% mutate(x4p9=X4/X9)
model_wf_4per9_log <- lm(log(y) ~ X3 + x4p9 + X6 + X7 + X8 , data=table_wf_resi)

# build X3/X4vX8*X9 eliminated log model
model_wf_34v89_log <- lm(log(y) ~ X3*X4 + X8*X9 + X6 + X7, data=table_wf_resi)

# build X3/X4vX8*X9 eliminated log model
model_wf_34v89v67_log <- lm(log(y) ~ X3*X4 + X8*X9 + X6*X7, data=table_wf_resi)

# build X8/X9vX4*X3 eliminated log model
table_wf_resi <- table_wf%>% mutate(x8p9=X8/X9)
model_wf_8per9v43_log <- lm(log(y) ~ x8p9 + X4*X3 + X6 + X7, data=table_wf_resi)

# build X6/7vX8/X9vX4X3 eliminated log model
table_wf_resi <- table_wf%>% mutate(x8p9=X8/X9,x6p7=X6/X7)
model_wf_6p7v8p9v43_log <- lm(log(y) ~ x8p9 + X4*X3 + x6p7, data=table_wf_resi)

# build X8/X9vX4*X3rmX7 eliminated log model
table_wf_resi <- table_wf%>% mutate(x8p9=X8/X9)
model_wf_8per9v43rm7_log <- lm(log(y) ~ x8p9 + X4*X3 + X6, data=table_wf_resi)

# build X8/X9vX4*X3vX6/X7 eliminated log model
table_wf_resi <- table_wf%>% mutate(x8p9=X8/X9)
model_wf_8per9v43rm7_log <- lm(log(y) ~ x8p9 + X4*X3 + X6, data=table_wf_resi)

huxreg(model_wf_8per9v43rm7_log, model_wf_8per9v43_log, model_wf_43rm1_log, model_wf_6p7v8p9v43_log, model_wf_34v89_log, model_wf_34v89v67_log)
(1) (2) (3) (4) (5) (6)
(Intercept) 1.765 *** 1.471 *** 2.084 *** 1.973 *** 2.526 *** 2.317 ** 
(0.365)    (0.377)    (0.424)    (0.400)    (0.557)    (0.712)   
x8p9 1.334 *** 1.307 ***          1.336 ***                  
(0.211)    (0.201)             (0.216)                     
X4 0.371 *** 0.302 *** 0.279 *** 0.321 *** 0.282 *** 0.303 ***
(0.040)    (0.052)    (0.058)    (0.045)    (0.057)    (0.073)   
X3 0.345 *** 0.338 *** 0.370 *** 0.327 *** 0.367 *** 0.382 ***
(0.061)    (0.058)    (0.065)    (0.063)    (0.064)    (0.072)   
X6 -0.293 *   -0.370 **  -0.303 *            -0.309 *   -0.125    
(0.113)    (0.114)    (0.125)             (0.124)    (0.401)   
X4:X3 -0.031 *** -0.025 **  -0.025 **  -0.025 **                   
(0.007)    (0.007)    (0.008)    (0.007)                     
X7          1.966     2.632 *            2.496 *   2.803 *  
         (1.007)    (1.127)             (1.121)    (1.306)   
X9                   -0.455 ***          -0.595 *** -0.593 ***
                  (0.092)             (0.147)    (0.150)   
X8                   0.617 ***          0.492 **  0.490 ** 
                  (0.112)             (0.152)    (0.155)   
x6p7                            -0.097 *                    
                           (0.040)                     
X3:X4                                     -0.026 **  -0.028 ** 
                                    (0.008)    (0.010)   
X8:X9                                     0.042     0.041    
                                    (0.035)    (0.035)   
X6:X7                                              -0.493    
                                             (1.016)   
N 30         30         30         30         30         30        
R2 0.964     0.969     0.963     0.963     0.966     0.966    
logLik -5.535     -3.233     -5.970     -5.980     -4.966     -4.790    
AIC 25.069     22.467     29.940     25.959     29.931     31.581    
*** p < 0.001; ** p < 0.01; * p < 0.05.
# build all log model
model_wf_all_log <- lm(log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + log(X7) + log(X8) + log(X9), data=table_wf)
ols_vif_tol(model_wf_all_log)
Variables Tolerance VIF
log(X1) 0.00608 164   
log(X2) 0.0865  11.6 
log(X3) 0.0816  12.3 
log(X4) 0.00767 130   
log(X5) 0.0885  11.3 
log(X6) 0.421   2.37
log(X7) 0.108   9.29
log(X8) 0.193   5.18
log(X9) 0.187   5.35
model_wf_aic_all_log <- stepAIC(model_wf_all_log)
## Start:  AIC=-84.46
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9)
## 
##           Df Sum of Sq    RSS     AIC
## - log(X7)  1    0.0027 0.9251 -86.371
## - log(X2)  1    0.0310 0.9534 -85.468
## - log(X4)  1    0.0370 0.9595 -85.277
## - log(X3)  1    0.0525 0.9750 -84.797
## - log(X5)  1    0.0574 0.9798 -84.647
## <none>                 0.9224 -84.459
## - log(X6)  1    0.2329 1.1553 -79.705
## - log(X1)  1    0.2712 1.1936 -78.727
## - log(X9)  1    3.4818 4.4043 -39.559
## - log(X8)  1    3.6487 4.5711 -38.443
## 
## Step:  AIC=-86.37
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X8) + log(X9)
## 
##           Df Sum of Sq    RSS     AIC
## - log(X4)  1    0.0370 0.9621 -87.193
## - log(X2)  1    0.0559 0.9810 -86.611
## <none>                 0.9251 -86.371
## - log(X3)  1    0.0777 1.0028 -85.953
## - log(X5)  1    0.0983 1.0234 -85.341
## - log(X6)  1    0.3174 1.2425 -79.522
## - log(X1)  1    0.3899 1.3150 -77.820
## - log(X9)  1    3.4793 4.4044 -41.558
## - log(X8)  1    3.6745 4.5996 -40.257
## 
## Step:  AIC=-87.19
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X5) + log(X6) + log(X8) + 
##     log(X9)
## 
##           Df Sum of Sq    RSS     AIC
## - log(X2)  1    0.0369 0.9990 -88.065
## <none>                 0.9621 -87.193
## - log(X5)  1    0.1419 1.1040 -85.067
## - log(X6)  1    0.2985 1.2607 -81.087
## - log(X3)  1    0.5087 1.4709 -76.460
## - log(X9)  1    3.4515 4.4137 -43.495
## - log(X8)  1    3.6420 4.6042 -42.227
## - log(X1)  1    3.8134 4.7755 -41.131
## 
## Step:  AIC=-88.07
## log(y) ~ log(X1) + log(X3) + log(X5) + log(X6) + log(X8) + log(X9)
## 
##           Df Sum of Sq     RSS     AIC
## <none>                  0.9990 -88.065
## - log(X5)  1    0.1087  1.1077 -86.967
## - log(X6)  1    0.3805  1.3795 -80.384
## - log(X3)  1    0.8252  1.8242 -72.001
## - log(X9)  1    3.4549  4.4539 -45.222
## - log(X8)  1    3.7305  4.7295 -43.421
## - log(X1)  1   17.5601 18.5592  -2.407
ols_vif_tol(model_wf_aic_all_log)
Variables Tolerance VIF
log(X1) 0.263 3.8 
log(X3) 0.603 1.66
log(X5) 0.22  4.55
log(X6) 0.71  1.41
log(X8) 0.201 4.99
log(X9) 0.191 5.22
# Interaction regression for all log  model
model_wf_all_log_inter <- lm(log(y) ~ (log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + log(X7) + log(X8) + log(X9))^2, data=table_wf)
model_wf_aic_all_log_inter <- stepAIC(model_wf_all_log_inter)
## Start:  AIC=-117.75
## log(y) ~ (log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9))^2
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X5) + 
##     log(X4):log(X6) + log(X4):log(X7) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X6) + log(X5):log(X7) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + 
##     log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X5) + 
##     log(X4):log(X6) + log(X4):log(X7) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X6) + log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + 
##     log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X5) + 
##     log(X4):log(X6) + log(X4):log(X7) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + 
##     log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X5) + 
##     log(X4):log(X6) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X5) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + 
##     log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X7) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + 
##     log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + 
##     log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X6) + 
##     log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + 
##     log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X5) + log(X3):log(X8) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X4) + log(X3):log(X8) + log(X3):log(X9) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + 
##     log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X7) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + 
##     log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + 
##     log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X6) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + 
##     log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X5) + log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X8) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X4) + 
##     log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X8) + log(X3):log(X9) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + 
##     log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X3) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + 
##     log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + 
##     log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X7) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + 
##     log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X6) + log(X1):log(X8) + 
##     log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X8) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X5) + log(X1):log(X8) + log(X1):log(X9) + 
##     log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X8) + log(X3):log(X9) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + 
##     log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X4) + log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + 
##     log(X2):log(X9) + log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + 
##     log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + 
##     log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
## 
## Step:  AIC=-117.75
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X8) + log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + 
##     log(X5):log(X8) + log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + 
##     log(X7):log(X8) + log(X7):log(X9) + log(X8):log(X9)
## 
##                   Df Sum of Sq      RSS     AIC
## - log(X3):log(X8)  1 0.0000903 0.097990 -119.72
## - log(X3):log(X9)  1 0.0003856 0.098286 -119.63
## - log(X7):log(X8)  1 0.0008771 0.098777 -119.48
## - log(X2):log(X9)  1 0.0009602 0.098860 -119.46
## - log(X2):log(X8)  1 0.0012711 0.099171 -119.36
## - log(X4):log(X9)  1 0.0013307 0.099231 -119.34
## - log(X6):log(X8)  1 0.0013914 0.099291 -119.33
## - log(X7):log(X9)  1 0.0014236 0.099324 -119.32
## - log(X5):log(X8)  1 0.0014490 0.099349 -119.31
## - log(X5):log(X9)  1 0.0015560 0.099456 -119.28
## - log(X4):log(X8)  1 0.0016138 0.099514 -119.26
## - log(X6):log(X9)  1 0.0016209 0.099521 -119.26
## - log(X1):log(X9)  1 0.0025113 0.100411 -118.99
## - log(X1):log(X2)  1 0.0025483 0.100448 -118.98
## - log(X8):log(X9)  1 0.0025636 0.100464 -118.97
## - log(X1):log(X8)  1 0.0029919 0.100892 -118.85
## <none>                         0.097900 -117.75
## - log(X1):log(X3)  1 0.0093915 0.107292 -117.00
## 
## Step:  AIC=-119.72
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9) + log(X8):log(X9)
## 
##                   Df Sum of Sq     RSS     AIC
## - log(X8):log(X9)  1  0.004379 0.10237 -120.41
## - log(X1):log(X2)  1  0.004551 0.10254 -120.36
## - log(X3):log(X9)  1  0.004640 0.10263 -120.33
## <none>                         0.09799 -119.72
## - log(X7):log(X8)  1  0.006852 0.10484 -119.69
## - log(X7):log(X9)  1  0.011626 0.10962 -118.36
## - log(X6):log(X8)  1  0.012314 0.11030 -118.17
## - log(X6):log(X9)  1  0.016183 0.11417 -117.14
## - log(X1):log(X9)  1  0.022498 0.12049 -115.52
## - log(X4):log(X9)  1  0.024248 0.12224 -115.09
## - log(X1):log(X8)  1  0.025269 0.12326 -114.84
## - log(X2):log(X9)  1  0.025677 0.12367 -114.74
## - log(X1):log(X3)  1  0.027330 0.12532 -114.34
## - log(X5):log(X9)  1  0.029018 0.12701 -113.94
## - log(X4):log(X8)  1  0.030440 0.12843 -113.61
## - log(X5):log(X8)  1  0.031030 0.12902 -113.47
## - log(X2):log(X8)  1  0.034946 0.13294 -112.57
## 
## Step:  AIC=-120.41
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X2) + log(X1):log(X3) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + 
##     log(X7):log(X9)
## 
##                   Df Sum of Sq     RSS     AIC
## - log(X1):log(X2)  1 0.0019105 0.10428 -121.86
## - log(X7):log(X8)  1 0.0042768 0.10665 -121.18
## <none>                         0.10237 -120.41
## - log(X7):log(X9)  1 0.0085205 0.11089 -120.01
## - log(X6):log(X8)  1 0.0088118 0.11118 -119.93
## - log(X6):log(X9)  1 0.0122126 0.11458 -119.03
## - log(X1):log(X9)  1 0.0181216 0.12049 -117.52
## - log(X4):log(X9)  1 0.0203055 0.12268 -116.98
## - log(X3):log(X9)  1 0.0205839 0.12295 -116.91
## - log(X1):log(X8)  1 0.0209330 0.12330 -116.83
## - log(X2):log(X9)  1 0.0213148 0.12368 -116.74
## - log(X1):log(X3)  1 0.0237155 0.12609 -116.16
## - log(X5):log(X9)  1 0.0249321 0.12730 -115.87
## - log(X4):log(X8)  1 0.0267336 0.12910 -115.45
## - log(X5):log(X8)  1 0.0270663 0.12944 -115.37
## - log(X2):log(X8)  1 0.0305761 0.13295 -114.57
## 
## Step:  AIC=-121.86
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X3) + log(X1):log(X8) + 
##     log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X9) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X8) + log(X7):log(X9)
## 
##                   Df Sum of Sq     RSS     AIC
## - log(X7):log(X8)  1 0.0055209 0.10980 -122.31
## <none>                         0.10428 -121.86
## - log(X7):log(X9)  1 0.0100914 0.11437 -121.08
## - log(X6):log(X8)  1 0.0111603 0.11544 -120.81
## - log(X6):log(X9)  1 0.0144674 0.11875 -119.96
## - log(X2):log(X9)  1 0.0220327 0.12631 -118.11
## - log(X1):log(X9)  1 0.0224953 0.12677 -118.00
## - log(X3):log(X9)  1 0.0225020 0.12678 -118.00
## - log(X4):log(X9)  1 0.0227039 0.12698 -117.95
## - log(X1):log(X3)  1 0.0232574 0.12754 -117.82
## - log(X5):log(X9)  1 0.0244629 0.12874 -117.53
## - log(X5):log(X8)  1 0.0257774 0.13006 -117.23
## - log(X1):log(X8)  1 0.0262811 0.13056 -117.11
## - log(X4):log(X8)  1 0.0296051 0.13389 -116.36
## - log(X2):log(X8)  1 0.0312323 0.13551 -116.00
## 
## Step:  AIC=-122.31
## log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + log(X5) + log(X6) + 
##     log(X7) + log(X8) + log(X9) + log(X1):log(X3) + log(X1):log(X8) + 
##     log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + log(X3):log(X9) + 
##     log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + log(X5):log(X9) + 
##     log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X9)
## 
##                   Df Sum of Sq     RSS      AIC
## <none>                         0.10980 -122.309
## - log(X3):log(X9)  1  0.020900 0.13070 -119.081
## - log(X1):log(X3)  1  0.021290 0.13109 -118.992
## - log(X7):log(X9)  1  0.023868 0.13367 -118.408
## - log(X6):log(X8)  1  0.030444 0.14024 -116.967
## - log(X5):log(X8)  1  0.032198 0.14200 -116.594
## - log(X1):log(X9)  1  0.034511 0.14431 -116.109
## - log(X5):log(X9)  1  0.036577 0.14638 -115.683
## - log(X6):log(X9)  1  0.057094 0.16689 -111.748
## - log(X1):log(X8)  1  0.058955 0.16876 -111.415
## - log(X2):log(X9)  1  0.067720 0.17752 -109.896
## - log(X4):log(X9)  1  0.086014 0.19581 -106.953
## - log(X2):log(X8)  1  0.117909 0.22771 -102.426
## - log(X4):log(X8)  1  0.199440 0.30924  -93.245
huxreg(model_wf_aic_log, model_wf_aic_all_log, model_wf_aic_log_inter, model_wf_aic_all_log_inter)
(1) (2) (3) (4)
(Intercept) 2.692 *** 0.571     -0.981     -16.027   
(0.445)    (3.360)    (1.520)    (13.947)  
X3 0.184 ***          0.493 ***        
(0.032)             (0.066)           
X4 0.109 ***          0.168 **         
(0.026)             (0.054)           
X6 -0.368 *                            
(0.146)                             
X7 4.085 **           2.515            
(1.213)             (1.258)           
X8 0.612 ***          0.586 ***        
(0.133)             (0.087)           
X9 -0.448 ***          -0.248 *          
(0.108)             (0.105)           
log(X1)          0.726 ***          1.158   
         (0.036)             (0.697)  
log(X3)          0.419 ***          1.666   
         (0.096)             (1.763)  
log(X5)          1.259              5.028   
         (0.796)             (3.473)  
log(X6)          -0.267 **           -0.158   
         (0.090)             (0.300)  
log(X8)          1.623 ***          44.919   
         (0.175)             (24.175)  
log(X9)          -1.375 ***          -37.066   
         (0.154)             (19.170)  
X1                   2.712 ***        
                  (0.330)           
X2                   -24.452 ***        
                  (5.631)           
X5                   0.039 *          
                  (0.016)           
X1:X3                   -0.416 ***        
                  (0.045)           
X1:X9                   -0.118 **         
                  (0.036)           
X2:X9                   4.643 **         
                  (1.449)           
X3:X8                   -0.051 **         
                  (0.014)           
X4:X8                   0.071 ***        
                  (0.015)           
X7:X9                   -0.995 *          
                  (0.344)           
log(X2)                            -0.733   
                           (0.338)  
log(X4)                            -2.394   
                           (2.963)  
log(X7)                            -0.408   
                           (0.518)  
log(X1):log(X3)                            0.823   
                           (0.707)  
log(X1):log(X8)                            1.242   
                           (0.641)  
log(X1):log(X9)                            -1.000   
                           (0.674)  
log(X2):log(X8)                            1.089 * 
                           (0.397)  
log(X2):log(X9)                            -0.710   
                           (0.341)  
log(X3):log(X9)                            0.411   
                           (0.356)  
log(X4):log(X8)                            -3.029 **
                           (0.849)  
log(X4):log(X9)                            2.174   
                           (0.929)  
log(X5):log(X8)                            -7.969   
                           (5.562)  
log(X5):log(X9)                            6.795   
                           (4.450)  
log(X6):log(X8)                            0.403   
                           (0.289)  
log(X6):log(X9)                            -0.506   
                           (0.265)  
log(X7):log(X9)                            0.464   
                           (0.376)  
N 30         30         30         30       
R2 0.947     0.986     0.994     0.998   
logLik -11.581     8.464     20.797     41.586   
AIC 39.161     -0.929     -9.594     -35.172   
*** p < 0.001; ** p < 0.01; * p < 0.05.
  • Additional Regression
# All Possible Regression for full log model #
k <- ols_step_all_possible(model_wf_full_log)
# k
plot(k)

# All Possible Regression for all log model #
k <- ols_step_all_possible(model_wf_all_log)
# k
plot(k)

# All Possible Regression for X4 eliminated model #
# k <- ols_step_all_possible(model_wf_rm4_log)
# k
plot(k)

# All Possible Regression for X1 eliminated model #
# k <- ols_step_all_possible(model_wf_rm1_log)
# k
plot(k)

# Stepwise Regression based on p values for full model#
k <- ols_step_both_p(model_wf_full_log)
## Stepwise Selection Method   
## ---------------------------
## 
## Candidate Terms: 
## 
## 1. X1 
## 2. X2 
## 3. X3 
## 4. X4 
## 5. X5 
## 6. X6 
## 7. X7 
## 8. X8 
## 9. X9 
## 
## We are selecting variables based on p value...
## 
## Variables Entered/Removed: 
## 
## - X4 added 
## - X3 added 
## - X7 added 
## 
## No more variables to be added/removed.
## 
## 
## Final Model Output 
## ------------------
## 
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.944       RMSE               0.549 
## R-Squared               0.890       Coef. Var          8.618 
## Adj. R-Squared          0.878       MSE                0.301 
## Pred R-Squared          0.854       MAE                0.414 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     63.565         3         21.188    70.378    0.0000 
## Residual        7.828        26          0.301                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                  Parameter Estimates                                  
## -------------------------------------------------------------------------------------
##       model     Beta    Std. Error    Std. Beta      t       Sig      lower    upper 
## -------------------------------------------------------------------------------------
## (Intercept)    2.872         0.547                 5.254    0.000     1.748    3.995 
##          X4    0.122         0.033        0.559    3.730    0.001     0.055    0.189 
##          X3    0.168         0.040        0.435    4.165    0.000     0.085    0.251 
##          X7    3.106         1.537        0.309    2.021    0.054    -0.053    6.266 
## -------------------------------------------------------------------------------------
k
## 
##                              Stepwise Selection Summary                              
## ------------------------------------------------------------------------------------
##                      Added/                   Adj.                                      
## Step    Variable    Removed     R-Square    R-Square     C(p)        AIC       RMSE     
## ------------------------------------------------------------------------------------
##    1       X4       addition       0.803       0.796    48.8550    68.4060    0.7087    
##    2       X3       addition       0.873       0.864    24.2130    57.2082    0.5792    
##    3       X7       addition       0.890       0.878    19.6670    54.8305    0.5487    
## ------------------------------------------------------------------------------------
plot(k)

# Stepwise AIC Regression for full model#
k<- ols_step_both_aic(model_wf_full_log)
## Stepwise Selection Method 
## -------------------------
## 
## Candidate Terms: 
## 
## 1 . X1 
## 2 . X2 
## 3 . X3 
## 4 . X4 
## 5 . X5 
## 6 . X6 
## 7 . X7 
## 8 . X8 
## 9 . X9 
## 
## 
## Variables Entered/Removed: 
## 
## - X4 added 
## - X3 added 
## - X7 added 
## - X8 added 
## - X9 added 
## - X6 added 
## 
## No more variables to be added or removed.
k
## 
## 
##                               Stepwise Summary                              
## --------------------------------------------------------------------------
## Variable     Method      AIC       RSS      Sum Sq     R-Sq      Adj. R-Sq 
## --------------------------------------------------------------------------
## X4          addition    68.406    14.063    57.330    0.80302      0.79599 
## X3          addition    57.208     9.057    62.335    0.87313      0.86373 
## X7          addition    54.830     7.828    63.565    0.89036      0.87771 
## X8          addition    54.522     7.248    64.144    0.89848      0.88223 
## X9          addition    44.504     4.856    66.537    0.93199      0.91782 
## X6          addition    39.161     3.801    67.591    0.94675      0.93286 
## --------------------------------------------------------------------------
plot(k)

# Stepwise Regression based on p values for X4 eliminated model#
# k <- ols_step_both_p(model_wf_rm4_log)
# k
# plot(k)

# Stepwise AIC Regression for X4 eliminated model#
# k<- ols_step_both_aic(model_wf_rm4_log)
# k
# plot(k)

# Stepwise Regression based on p values for X1 eliminated model#
# k <- ols_step_both_p(model_wf_rm1_log)
# k
# plot(k)

# Stepwise AIC Regression for X1 eliminated model#
# k<- ols_step_both_aic(model_wf_rm1_log)
# k
# plot(k)
#Lack of Fit F Test

ols_pure_error_anova(lm(y~X1, data = table_wf))
ols_pure_error_anova(lm(y~X4, data = table_wf))

alias(lm(y ~ as.factor(X3) + as.factor(X4) + as.factor(X5) + as.factor(X6) + as.factor(X7), data=table_wf))

alias(lm(y ~ as.factor(X1) + as.factor(X8) , data=table_wf))

alias(lm(y ~ as.factor(X4) + as.factor(X9) , data=table_wf))

alias(lm(y ~ as.factor(X3) + as.factor(X6) + as.factor(X7) + as.factor(X8) + as.factor(X9) , data=table_wf))

(d) (4) Forward selection

Stepwise Forward Regression for full model

# Stepwise Forward Regression based on p values (use α=0.15) #
ols_step_forward_p(model_wf_full_log, penter = 0.15)
## Forward Selection Method    
## ---------------------------
## 
## Candidate Terms: 
## 
## 1. X1 
## 2. X2 
## 3. X3 
## 4. X4 
## 5. X5 
## 6. X6 
## 7. X7 
## 8. X8 
## 9. X9 
## 
## We are selecting variables based on p value...
## 
## Variables Entered: 
## 
## - X4 
## - X3 
## - X7 
## 
## No more variables to be added.
## 
## Final Model Output 
## ------------------
## 
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.944       RMSE               0.549 
## R-Squared               0.890       Coef. Var          8.618 
## Adj. R-Squared          0.878       MSE                0.301 
## Pred R-Squared          0.854       MAE                0.414 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     63.565         3         21.188    70.378    0.0000 
## Residual        7.828        26          0.301                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                  Parameter Estimates                                  
## -------------------------------------------------------------------------------------
##       model     Beta    Std. Error    Std. Beta      t       Sig      lower    upper 
## -------------------------------------------------------------------------------------
## (Intercept)    2.872         0.547                 5.254    0.000     1.748    3.995 
##          X4    0.122         0.033        0.559    3.730    0.001     0.055    0.189 
##          X3    0.168         0.040        0.435    4.165    0.000     0.085    0.251 
##          X7    3.106         1.537        0.309    2.021    0.054    -0.053    6.266 
## -------------------------------------------------------------------------------------
## 
##                            Selection Summary                             
## ------------------------------------------------------------------------
##         Variable                  Adj.                                      
## Step    Entered     R-Square    R-Square     C(p)        AIC       RMSE     
## ------------------------------------------------------------------------
##    1    X4            0.8030      0.7960    48.8552    68.4060    0.7087    
##    2    X3            0.8731      0.8637    24.2129    57.2082    0.5792    
##    3    X7            0.8904      0.8777    19.6668    54.8305    0.5487    
## ------------------------------------------------------------------------
# Stepwise AIC Forward Regression #
ols_step_forward_aic(model_wf_full_log)
## Forward Selection Method 
## ------------------------
## 
## Candidate Terms: 
## 
## 1 . X1 
## 2 . X2 
## 3 . X3 
## 4 . X4 
## 5 . X5 
## 6 . X6 
## 7 . X7 
## 8 . X8 
## 9 . X9 
## 
## 
## Variables Entered: 
## 
## - X4 
## - X3 
## - X7 
## - X8 
## - X9 
## - X6 
## 
## No more variables to be added.
## 
##                        Selection Summary                        
## ---------------------------------------------------------------
## Variable      AIC      Sum Sq     RSS       R-Sq      Adj. R-Sq 
## ---------------------------------------------------------------
## X4           68.406    57.330    14.063    0.80302      0.79599 
## X3           57.208    62.335     9.057    0.87313      0.86373 
## X7           54.830    63.565     7.828    0.89036      0.87771 
## X8           54.522    64.144     7.248    0.89848      0.88223 
## X9           44.504    66.537     4.856    0.93199      0.91782 
## X6           39.161    67.591     3.801    0.94675      0.93286 
## ---------------------------------------------------------------

Stepwise Forward Regression for X4 eliminated model

# Stepwise Forward Regression based on p values (use α=0.15) #
ols_step_forward_p(model_wf_rm4_log, penter = 0.15)
# Stepwise AIC Forward Regression #
ols_step_forward_aic(model_wf_rm4_log)

Stepwise Forward Regression for X1 eliminated model

# Stepwise Forward Regression based on p values (use α=0.15) #
ols_step_forward_p(model_wf_rm1_log, penter = 0.15)
# Stepwise AIC Forward Regression #
ols_step_forward_aic(model_wf_rm1_log)

(d) (5) Backward selection

Stepwise Backward Regression for full model

# Stepwise Backward Regression based on p values (use α=0.05) #
ols_step_backward_p(model_wf_full_log, penter = 0.05)
## Backward Elimination Method 
## ---------------------------
## 
## Candidate Terms: 
## 
## 1 . X1 
## 2 . X2 
## 3 . X3 
## 4 . X4 
## 5 . X5 
## 6 . X6 
## 7 . X7 
## 8 . X8 
## 9 . X9 
## 
## We are eliminating variables based on p value...
## 
## Variables Removed: 
## 
## - X1 
## - X2 
## - X5 
## 
## No more variables satisfy the condition of p value = 0.3
## 
## 
## Final Model Output 
## ------------------
## 
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.973       RMSE               0.407 
## R-Squared               0.947       Coef. Var          6.385 
## Adj. R-Squared          0.933       MSE                0.165 
## Pred R-Squared          0.908       MAE                0.273 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     67.591         6         11.265     68.16    0.0000 
## Residual        3.801        23          0.165                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                   Parameter Estimates                                    
## ----------------------------------------------------------------------------------------
##       model      Beta    Std. Error    Std. Beta      t        Sig      lower     upper 
## ----------------------------------------------------------------------------------------
## (Intercept)     2.692         0.445                  6.046    0.000     1.771     3.613 
##          X3     0.184         0.032        0.476     5.698    0.000     0.117     0.251 
##          X4     0.109         0.026        0.499     4.244    0.000     0.056     0.162 
##          X6    -0.368         0.146       -0.133    -2.526    0.019    -0.669    -0.066 
##          X7     4.085         1.213        0.406     3.367    0.003     1.575     6.595 
##          X8     0.612         0.133        0.493     4.614    0.000     0.337     0.886 
##          X9    -0.448         0.108       -0.450    -4.135    0.000    -0.672    -0.224 
## ----------------------------------------------------------------------------------------
## 
## 
##                           Elimination Summary                           
## -----------------------------------------------------------------------
##         Variable                  Adj.                                     
## Step    Removed     R-Square    R-Square     C(p)       AIC       RMSE     
## -----------------------------------------------------------------------
##    1    X1            0.9474      0.9273    8.0021    42.8146    0.4230    
##    2    X2            0.9472      0.9304    6.0604    40.9019    0.4139    
##    3    X5            0.9468      0.9329    4.2345    39.1611    0.4065    
## -----------------------------------------------------------------------
# Stepwise AIC Backward Regression #
ols_step_backward_aic(model_wf_full_log)
## Backward Elimination Method 
## ---------------------------
## 
## Candidate Terms: 
## 
## 1 . X1 
## 2 . X2 
## 3 . X3 
## 4 . X4 
## 5 . X5 
## 6 . X6 
## 7 . X7 
## 8 . X8 
## 9 . X9 
## 
## 
## Variables Removed: 
## 
## - X1 
## - X2 
## - X5 
## 
## No more variables to be removed.
## 
## 
##                   Backward Elimination Summary                   
## ---------------------------------------------------------------
## Variable       AIC       RSS     Sum Sq     R-Sq      Adj. R-Sq 
## ---------------------------------------------------------------
## Full Model    44.811    3.757    67.635    0.94737      0.92369 
## X1            42.815    3.758    67.635    0.94737      0.92731 
## X2            40.902    3.769    67.624    0.94721      0.93042 
## X5            39.161    3.801    67.591    0.94675      0.93286 
## ---------------------------------------------------------------

Stepwise Backward Regression for X4 eliminated model

# Stepwise Backward Regression based on p values (use α=0.05) #
ols_step_backward_p(model_wf_rm4_log, penter = 0.05)
# Stepwise AIC Backward Regression #
ols_step_backward_aic(model_wf_rm4_log)

Stepwise Backward Regression for X1 eliminated model

# Stepwise Backward Regression based on p values (use α=0.05) #
ols_step_backward_p(model_wf_rm1_log, penter = 0.05)
# Stepwise AIC Backward Regression #
ols_step_backward_aic(model_wf_rm1_log)

(d) (6) Best Subset Regression

# For full model #
k <- ols_step_best_subset(model_wf_full_log)
k
mindex n predictors rsquare adjr predrsq cp aic sbic sbc msep fpe apc hsp
1 1 X4 0.803 0.796 0.772 48.9  68.4 -19.8 72.6 0.538 0.536 0.225  0.0186 
2 2 X3 X4 0.873 0.864 0.844 24.2  57.2 -30.5 62.8 0.373 0.369 0.155  0.0129 
3 3 X3 X4 X7 0.89  0.878 0.854 19.7  54.8 -32.7 61.8 0.348 0.341 0.143  0.012  
4 4 X1 X4 X8 X9 0.921 0.908 0.886 10.1  47   -38.1 55.4 0.272 0.264 0.111  0.00941
5 5 X3 X4 X7 X8 X9 0.932 0.918 0.892 7.85 44.5 -38.8 54.3 0.255 0.243 0.102  0.0088 
6 6 X3 X4 X6 X7 X8 X9 0.947 0.933 0.908 4.23 39.2 -39.7 50.4 0.217 0.204 0.0857 0.00751
7 7 X3 X4 X5 X6 X7 X8 X9 0.947 0.93  0.902 6.06 40.9 -36.8 53.5 0.236 0.217 0.0912 0.00816
8 8 X2 X3 X4 X5 X6 X7 X8 X9 0.947 0.927 0.896 8    42.8 -33.8 56.8 0.259 0.233 0.0977 0.00895
9 9 X1 X2 X3 X4 X5 X6 X7 X8 X9 0.947 0.924 0.886 10    44.8 -30.8 60.2 0.286 0.25  0.105  0.00989
plot(k)

# For X4 eliminated model #
# k <- ols_step_best_subset(model_wf_rm4_log)
# k
# plot(k)

# For X1 eliminated model #
# k <- ols_step_best_subset(model_wf_rm1_log)
# k
# plot(k)

(d) (7) Models Comparison

  • Model 437896
# build model 437896
model_wf_437896_log <- lm(log(y) ~ X4 + X3 + X7 + X8 + X9 + X6, data=table_wf)
ols_regress(model_wf_437896_log)
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.973       RMSE               0.407 
## R-Squared               0.947       Coef. Var          6.385 
## Adj. R-Squared          0.933       MSE                0.165 
## Pred R-Squared          0.908       MAE                0.273 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     67.591         6         11.265     68.16    0.0000 
## Residual        3.801        23          0.165                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                   Parameter Estimates                                    
## ----------------------------------------------------------------------------------------
##       model      Beta    Std. Error    Std. Beta      t        Sig      lower     upper 
## ----------------------------------------------------------------------------------------
## (Intercept)     2.692         0.445                  6.046    0.000     1.771     3.613 
##          X4     0.109         0.026        0.499     4.244    0.000     0.056     0.162 
##          X3     0.184         0.032        0.476     5.698    0.000     0.117     0.251 
##          X7     4.085         1.213        0.406     3.367    0.003     1.575     6.595 
##          X8     0.612         0.133        0.493     4.614    0.000     0.337     0.886 
##          X9    -0.448         0.108       -0.450    -4.135    0.000    -0.672    -0.224 
##          X6    -0.368         0.146       -0.133    -2.526    0.019    -0.669    -0.066 
## ----------------------------------------------------------------------------------------
# Collinearity Diagnostics #
ols_vif_tol(model_wf_437896_log)
Variables Tolerance VIF
X4 0.167 5.97
X3 0.332 3.01
X7 0.159 6.28
X8 0.202 4.94
X9 0.195 5.12
X6 0.839 1.19
#Model Fit Assessment
ols_plot_diagnostics(model_wf_437896_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_437896_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9837263
# Residual Normality Test
ols_test_normality(model_wf_437896_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9728         0.6175 
## Kolmogorov-Smirnov        0.0997         0.8982 
## Cramer-von Mises          4.8429         0.0000 
## Anderson-Darling          0.2996         0.5612 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_437896_log)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_437896_log)

  • Model 437
# build model 437
model_wf_437_log <- lm(log(y) ~ X4 + X3 + X7, data=table_wf)
ols_regress(model_wf_437_log)
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.944       RMSE               0.549 
## R-Squared               0.890       Coef. Var          8.618 
## Adj. R-Squared          0.878       MSE                0.301 
## Pred R-Squared          0.854       MAE                0.414 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     63.565         3         21.188    70.378    0.0000 
## Residual        7.828        26          0.301                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                  Parameter Estimates                                  
## -------------------------------------------------------------------------------------
##       model     Beta    Std. Error    Std. Beta      t       Sig      lower    upper 
## -------------------------------------------------------------------------------------
## (Intercept)    2.872         0.547                 5.254    0.000     1.748    3.995 
##          X4    0.122         0.033        0.559    3.730    0.001     0.055    0.189 
##          X3    0.168         0.040        0.435    4.165    0.000     0.085    0.251 
##          X7    3.106         1.537        0.309    2.021    0.054    -0.053    6.266 
## -------------------------------------------------------------------------------------
# Collinearity Diagnostics #
ols_vif_tol(model_wf_437_log)
Variables Tolerance VIF
X4 0.188 5.32
X3 0.386 2.59
X7 0.181 5.53
#Model Fit Assessment
ols_plot_diagnostics(model_wf_437_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_437_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9856766
# Residual Normality Test
ols_test_normality(model_wf_437_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9765         0.7267 
## Kolmogorov-Smirnov        0.1033         0.8736 
## Cramer-von Mises          3.1908         0.0000 
## Anderson-Darling          0.3511         0.4469 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_437_log)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_437_log)

  • Model 137689
# build model 137689
model_wf_137689_log <- lm(log(y) ~ X1 + X3 + X7 + X6 + X8 + X9, data=table_wf)
ols_regress(model_wf_137689_log)
##                         Model Summary                         
## -------------------------------------------------------------
## R                       0.971       RMSE               0.421 
## R-Squared               0.943       Coef. Var          6.618 
## Adj. R-Squared          0.928       MSE                0.178 
## Pred R-Squared          0.900       MAE                0.292 
## -------------------------------------------------------------
##  RMSE: Root Mean Square Error 
##  MSE: Mean Square Error 
##  MAE: Mean Absolute Error 
## 
##                                ANOVA                                
## -------------------------------------------------------------------
##                Sum of                                              
##               Squares        DF    Mean Square      F         Sig. 
## -------------------------------------------------------------------
## Regression     67.310         6         11.218    63.195    0.0000 
## Residual        4.083        23          0.178                     
## Total          71.393        29                                    
## -------------------------------------------------------------------
## 
##                                   Parameter Estimates                                    
## ----------------------------------------------------------------------------------------
##       model      Beta    Std. Error    Std. Beta      t        Sig      lower     upper 
## ----------------------------------------------------------------------------------------
## (Intercept)     2.307         0.410                  5.623    0.000     1.458     3.156 
##          X1     0.207         0.053        0.368     3.897    0.001     0.097     0.317 
##          X3     0.263         0.022        0.680    11.944    0.000     0.217     0.308 
##          X7     5.453         1.002        0.542     5.442    0.000     3.380     7.525 
##          X6    -0.532         0.144       -0.192    -3.688    0.001    -0.831    -0.234 
##          X8     0.613         0.137        0.495     4.462    0.000     0.329     0.897 
##          X9    -0.433         0.112       -0.435    -3.864    0.001    -0.665    -0.201 
## ----------------------------------------------------------------------------------------
# Collinearity Diagnostics #
ols_vif_tol(model_wf_137689_log)
Variables Tolerance VIF
X1 0.279 3.58
X3 0.768 1.3 
X7 0.251 3.99
X6 0.917 1.09
X8 0.202 4.94
X9 0.196 5.1 
#Model Fit Assessment
ols_plot_diagnostics(model_wf_137689_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_137689_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.988106
# Residual Normality Test
ols_test_normality(model_wf_137689_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9769         0.7382 
## Kolmogorov-Smirnov        0.0771         0.9881 
## Cramer-von Mises          4.4689         0.0000 
## Anderson-Darling          0.1644         0.9350 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_137689_log)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_137689_log)

Final models

  • Check model aic_log
summary(model_wf_aic_log)
## 
## Call:
## lm(formula = log(y) ~ X3 + X4 + X6 + X7 + X8 + X9, data = table_wf)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.93767 -0.23482 -0.03261  0.19331  0.73570 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  2.69180    0.44521   6.046 3.63e-06 ***
## X3           0.18384    0.03227   5.698 8.41e-06 ***
## X4           0.10905    0.02569   4.244 0.000306 ***
## X6          -0.36752    0.14552  -2.526 0.018898 *  
## X7           4.08497    1.21317   3.367 0.002662 ** 
## X8           0.61161    0.13256   4.614 0.000122 ***
## X9          -0.44764    0.10826  -4.135 0.000402 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.4065 on 23 degrees of freedom
## Multiple R-squared:  0.9468, Adjusted R-squared:  0.9329 
## F-statistic: 68.16 on 6 and 23 DF,  p-value: 1.717e-13
Anova(model_wf_aic_log)
Sum Sq Df F value Pr(>F)
5.37 1 32.5  8.41e-06
2.98 1 18    0.000306
1.05 1 6.38 0.0189  
1.87 1 11.3  0.00266 
3.52 1 21.3  0.000122
2.83 1 17.1  0.000402
3.8  23            
# Collinearity Diagnostics #
ols_vif_tol(model_wf_aic_log)
Variables Tolerance VIF
X3 0.332 3.01
X4 0.167 5.97
X6 0.839 1.19
X7 0.159 6.28
X8 0.202 4.94
X9 0.195 5.12
#Model Fit Assessment
ols_plot_diagnostics(model_wf_aic_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_aic_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9837263
# Residual Normality Test
ols_test_normality(model_wf_aic_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9728         0.6175 
## Kolmogorov-Smirnov        0.0997         0.8982 
## Cramer-von Mises          4.8429         0.0000 
## Anderson-Darling          0.2996         0.5612 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_aic_log)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_aic_log)

  • Check model aic_log_inter
summary(model_wf_aic_log_inter)
## 
## Call:
## lm(formula = log(y) ~ X1 + X2 + X3 + X4 + X5 + X7 + X8 + X9 + 
##     X1:X3 + X1:X9 + X2:X9 + X3:X8 + X4:X8 + X7:X9, data = table_wf)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.22469 -0.06872  0.00151  0.04884  0.42819 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  -0.98138    1.51954  -0.646 0.528142    
## X1            2.71181    0.32987   8.221 6.15e-07 ***
## X2          -24.45201    5.63070  -4.343 0.000580 ***
## X3            0.49272    0.06581   7.487 1.93e-06 ***
## X4            0.16780    0.05415   3.099 0.007335 ** 
## X5            0.03903    0.01606   2.430 0.028137 *  
## X7            2.51458    1.25764   1.999 0.064011 .  
## X8            0.58605    0.08651   6.775 6.26e-06 ***
## X9           -0.24828    0.10461  -2.373 0.031416 *  
## X1:X3        -0.41634    0.04505  -9.242 1.40e-07 ***
## X1:X9        -0.11820    0.03621  -3.264 0.005231 ** 
## X2:X9         4.64270    1.44938   3.203 0.005925 ** 
## X3:X8        -0.05085    0.01367  -3.721 0.002050 ** 
## X4:X8         0.07075    0.01481   4.777 0.000244 ***
## X7:X9        -0.99460    0.34359  -2.895 0.011113 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.1711 on 15 degrees of freedom
## Multiple R-squared:  0.9939, Adjusted R-squared:  0.9881 
## F-statistic: 173.2 on 14 and 15 DF,  p-value: 5.77e-14
Anova(model_wf_aic_log_inter)
Sum Sq Df F value Pr(>F)
0.586   1 20     0.000446
0.277   1 9.48  0.00764 
0.00823 1 0.281 0.604   
2.99    1 102     4.36e-08
0.173   1 5.9   0.0281  
0.00974 1 0.333 0.573   
1.88    1 64.3   8.37e-07
2.59    1 88.4   1.12e-07
2.5     1 85.4   1.4e-07 
0.312   1 10.7   0.00523 
0.3     1 10.3   0.00592 
0.405   1 13.8   0.00205 
0.668   1 22.8   0.000244
0.245   1 8.38  0.0111  
0.439   15             
# Collinearity Diagnostics #
ols_vif_tol(model_wf_aic_log_inter)
Variables Tolerance VIF
X1 0.0012  835  
X2 0.032   31.3
X3 0.0141  70.8
X4 0.00668 150  
X5 0.0824  12.1
X7 0.0262  38.1
X8 0.0842  11.9
X9 0.0371  27  
X1:X3 0.00165 605  
X1:X9 0.00557 179  
X2:X9 0.0287  34.9
X3:X8 0.024   41.7
X4:X8 0.00579 173  
X7:X9 0.0111  90.3
#Model Fit Assessment
ols_plot_diagnostics(model_wf_aic_log_inter)

# Part & Partial Correlations
ols_test_correlation(model_wf_aic_log_inter) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9380444
# Residual Normality Test
ols_test_normality(model_wf_aic_log_inter) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.898          0.0075 
## Kolmogorov-Smirnov        0.1576         0.4037 
## Cramer-von Mises          8.0824         0.0000 
## Anderson-Darling          0.8459         0.0259 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_aic_log_inter)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_aic_log_inter)

  • Check model all log
summary(model_wf_aic_all_log)
## 
## Call:
## lm(formula = log(y) ~ log(X1) + log(X3) + log(X5) + log(X6) + 
##     log(X8) + log(X9), data = table_wf)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.67722 -0.08003  0.01102  0.13879  0.25715 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  0.57120    3.36000   0.170  0.86650    
## log(X1)      0.72550    0.03608  20.107 4.31e-16 ***
## log(X3)      0.41866    0.09605   4.359  0.00023 ***
## log(X5)      1.25873    0.79566   1.582  0.12731    
## log(X6)     -0.26702    0.09022  -2.960  0.00702 ** 
## log(X8)      1.62253    0.17508   9.267 3.15e-09 ***
## log(X9)     -1.37489    0.15416  -8.919 6.33e-09 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.2084 on 23 degrees of freedom
## Multiple R-squared:  0.986,  Adjusted R-squared:  0.9824 
## F-statistic: 270.1 on 6 and 23 DF,  p-value: < 2.2e-16
Anova(model_wf_aic_all_log)
Sum Sq Df F value Pr(>F)
17.6   1 404    4.31e-16
0.825 1 19    0.00023 
0.109 1 2.5  0.127   
0.38  1 8.76 0.00702 
3.73  1 85.9  3.15e-09
3.45  1 79.5  6.33e-09
0.999 23            
# Collinearity Diagnostics #
ols_vif_tol(model_wf_aic_all_log)
Variables Tolerance VIF
log(X1) 0.263 3.8 
log(X3) 0.603 1.66
log(X5) 0.22  4.55
log(X6) 0.71  1.41
log(X8) 0.201 4.99
log(X9) 0.191 5.22
#Model Fit Assessment
ols_plot_diagnostics(model_wf_aic_all_log)

# Part & Partial Correlations
ols_test_correlation(model_wf_aic_all_log) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9278999
# Residual Normality Test
ols_test_normality(model_wf_aic_all_log) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.8746         0.0021 
## Kolmogorov-Smirnov        0.0964         0.9180 
## Cramer-von Mises          7.0221         0.0000 
## Anderson-Darling          0.7277         0.0516 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_aic_all_log)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_aic_all_log)

  • Check model all log interaction
summary(model_wf_aic_all_log_inter)
## 
## Call:
## lm(formula = log(y) ~ log(X1) + log(X2) + log(X3) + log(X4) + 
##     log(X5) + log(X6) + log(X7) + log(X8) + log(X9) + log(X1):log(X3) + 
##     log(X1):log(X8) + log(X1):log(X9) + log(X2):log(X8) + log(X2):log(X9) + 
##     log(X3):log(X9) + log(X4):log(X8) + log(X4):log(X9) + log(X5):log(X8) + 
##     log(X5):log(X9) + log(X6):log(X8) + log(X6):log(X9) + log(X7):log(X9), 
##     data = table_wf)
## 
## Residuals:
##       Min        1Q    Median        3Q       Max 
## -0.114032 -0.038669 -0.003953  0.026220  0.160039 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)   
## (Intercept)     -16.0272    13.9469  -1.149  0.28823   
## log(X1)           1.1577     0.6966   1.662  0.14045   
## log(X2)          -0.7328     0.3383  -2.166  0.06698 . 
## log(X3)           1.6656     1.7630   0.945  0.37625   
## log(X4)          -2.3936     2.9633  -0.808  0.44581   
## log(X5)           5.0279     3.4728   1.448  0.19094   
## log(X6)          -0.1583     0.3004  -0.527  0.61463   
## log(X7)          -0.4075     0.5183  -0.786  0.45748   
## log(X8)          44.9189    24.1746   1.858  0.10550   
## log(X9)         -37.0656    19.1702  -1.934  0.09443 . 
## log(X1):log(X3)   0.8234     0.7067   1.165  0.28218   
## log(X1):log(X8)   1.2421     0.6407   1.939  0.09372 . 
## log(X1):log(X9)  -0.9995     0.6739  -1.483  0.18156   
## log(X2):log(X8)   1.0890     0.3972   2.742  0.02885 * 
## log(X2):log(X9)  -0.7095     0.3415  -2.078  0.07633 . 
## log(X3):log(X9)   0.4112     0.3562   1.154  0.28626   
## log(X4):log(X8)  -3.0288     0.8494  -3.566  0.00915 **
## log(X4):log(X9)   2.1744     0.9286   2.342  0.05172 . 
## log(X5):log(X8)  -7.9687     5.5620  -1.433  0.19504   
## log(X5):log(X9)   6.7951     4.4498   1.527  0.17059   
## log(X6):log(X8)   0.4027     0.2891   1.393  0.20622   
## log(X6):log(X9)  -0.5057     0.2651  -1.908  0.09807 . 
## log(X7):log(X9)   0.4637     0.3759   1.234  0.25719   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.1252 on 7 degrees of freedom
## Multiple R-squared:  0.9985, Adjusted R-squared:  0.9936 
## F-statistic: 206.6 on 22 and 7 DF,  p-value: 7.713e-08
Anova(model_wf_aic_all_log_inter)
Sum Sq Df F value Pr(>F)
0.281    1 17.9    0.00386 
0.00972  1 0.619  0.457   
0.0102   1 0.648  0.447   
0.000172 1 0.0109 0.92    
0.0311   1 1.98   0.202   
0.068    1 4.33   0.0759  
0.00234  1 0.149  0.711   
1.7      1 108      1.65e-05
1.73     1 111      1.53e-05
0.0213   1 1.36   0.282   
0.059    1 3.76   0.0937  
0.0345   1 2.2    0.182   
0.118    1 7.52   0.0288  
0.0677   1 4.32   0.0763  
0.0209   1 1.33   0.286   
0.199    1 12.7    0.00915 
0.086    1 5.48   0.0517  
0.0322   1 2.05   0.195   
0.0366   1 2.33   0.171   
0.0304   1 1.94   0.206   
0.0571   1 3.64   0.0981  
0.0239   1 1.52   0.257   
0.11     7              
# Collinearity Diagnostics #
ols_vif_tol(model_wf_aic_all_log_inter)
Variables Tolerance VIF
log(X1) 0.000255 3.92e+03
log(X2) 0.00469  213       
log(X3) 0.000646 1.55e+03
log(X4) 4.36e-05 2.29e+04
log(X5) 0.00416  240       
log(X6) 0.0231   43.2     
log(X7) 0.00911  110       
log(X8) 3.8e-06  2.63e+05
log(X9) 4.47e-06 2.24e+05
log(X1):log(X3) 0.000108 9.3e+03 
log(X1):log(X8) 0.000275 3.64e+03
log(X1):log(X9) 0.000202 4.94e+03
log(X2):log(X8) 0.000851 1.18e+03
log(X2):log(X9) 0.000809 1.24e+03
log(X3):log(X9) 0.00282  355       
log(X4):log(X8) 0.000283 3.53e+03
log(X4):log(X9) 0.00019  5.25e+03
log(X5):log(X8) 4.17e-06 2.4e+05 
log(X5):log(X9) 4.82e-06 2.07e+05
log(X6):log(X8) 0.0203   49.4     
log(X6):log(X9) 0.0191   52.3     
log(X7):log(X9) 0.00632  158       
#Model Fit Assessment
ols_plot_diagnostics(model_wf_aic_all_log_inter)

# Part & Partial Correlations
ols_test_correlation(model_wf_aic_all_log_inter) # Correlation between observed residuals and expected residuals under normality.
## [1] 0.9830352
# Residual Normality Test
ols_test_normality(model_wf_aic_all_log_inter) # Test for detecting violation of normality assumption. #If p-value is bigger, then no problem of non-normality #
## -----------------------------------------------
##        Test             Statistic       pvalue  
## -----------------------------------------------
## Shapiro-Wilk              0.9695         0.5254 
## Kolmogorov-Smirnov        0.1176         0.7579 
## Cramer-von Mises          8.8358         0.0000 
## Anderson-Darling          0.3654         0.4134 
## -----------------------------------------------
# Variable Contributions
ols_plot_added_variable(model_wf_aic_all_log_inter)

# Residual Plus Component Plot
ols_plot_comp_plus_resid(model_wf_aic_all_log_inter)

# Check PRESS Statistic
ols_press(model_wf_full)
## [1] 15880486
ols_press(model_wf_full_log)
## [1] 8.136733
ols_press(model_wf_437896_log)
## [1] 6.538275
ols_press(model_wf_437_log)
## [1] 10.43262
ols_press(model_wf_137689_log)
## [1] 7.114336
ols_press(model_wf_43rm1_log)
## [1] 4.962623
ols_press(model_wf_8per9v43_log)
## [1] 3.281548
ols_press(model_wf_aic_log)
## [1] 6.538275
ols_press(model_wf_aic_log_inter)
## [1] 1.336618
ols_press(model_wf_aic_all_log)
## [1] 1.793104
ols_press(model_wf_aic_all_log_inter)
## [1] 21.42167
# prediction power
ols_pred_rsq(model_wf_437896_log)
## [1] 0.908418
ols_pred_rsq(model_wf_137689_log)
## [1] 0.900349
ols_pred_rsq(model_wf_43rm1_log)
## [1] 0.9304882
ols_pred_rsq(model_wf_8per9v43_log)
## [1] 0.9540352
ols_pred_rsq(model_wf_aic_log)
## [1] 0.908418
ols_pred_rsq(model_wf_aic_log_inter)
## [1] 0.9812779
ols_pred_rsq(model_wf_aic_all_log)
## [1] 0.9748839
ols_pred_rsq(model_wf_aic_all_log_inter)
## [1] 0.6999453

library(texreg)
# Pretty print regression results on screen
lm(mpg ~ wt, data=my_df) %>% screenreg
texreg::screenreg(l=list(model_2_7_8))

# visualizng
library(GGally)
ggpairs(data=table_b1[c(1,3,8,9)])

# Correlation
cor(table_b1)

# Half correlation matrix:
library(corrr)
mtcars %>% correlate() %>% shave() %>% fashion()
# Visulize correlation matrix:
mtcars %>% correlate() %>% shave() %>% rplot()

# Scatterplot Matrix
mtcars[1:6] %>% plot
# Better looking version
library(ggfortify)
model_2_7_8 %>% autoplot()

# Confidence interval of coefficients
lm(mpg ~ wt + cyl, data=mtcars) %>% confint()

# Hypothesis testing of nested models
lm_mpg_wt <- lm(mpg ~ wt, data=mtcars)
lm_mpg_wt.cyl <- lm(mpg ~ wt + cyl, data=mtcars)
anova(lm_mpg_wt, lm_mpg_wt.cyl)

# convert mpg to kilometers per liter
mtcars %>% mutate(kmpl = mpg * 0.425144) %>% select(mpg, kmpl) %>% filter(mpg > 20)
nrow()
mtcars %>% group_by(am) %>% 
  summarize(n=n(),
            mean_mpg=mean(mpg),
            sd_mpg=sd(mpg),
            min_mpg=min(mpg),
            max_mpg=max(mpg)
mtcars %>% arrange(desc(mpg))

# mean & sd
mtcars %>% summarize(am_mean=mean(am), am_sd=sd(am))
# Frequencies by categories
mtcars %>% group_by(am) %>% tally

# Assume we want to combine LA + SD to Southern CA and Bay Area and Sacramento
## as Northern CA
(californiatod <- californiatod %>% 
  mutate(transit_level=case_when(
    transit>0.4~"high",
    transit>0.2~"medium",
    TRUE ~ "low")))


## General linear F test
fit_R <- lm(mpg ~ wt, data=mtcars)
fit_F <- lm(mpg ~ wt + cyl, data=mtcars)
anova(fit_R, fit_F)

SSE_R <- resid(fit_R)^2 %>% sum
SSE_F <- resid(fit_F)^2 %>% sum
df_R <- df.residual(fit_R)
df_F <- df.residual(fit_F)
F_val <- ((SSE_R - SSE_F)/(df_R - df_F))/(SSE_F/df_F)

# Look up the critical F value for alpha=0.05
alpha <- 0.05
qf(alpha, (df_R - df_F), df_F, lower.tail=F)
# Alternatively, find the p-value corresponding to our F_val
pf(F_val, (df_R - df_F), df_F, lower.tail=F)
n <- nrow(mtcars)          # number of observations
k <- length(coef(fit_R))   # number of coefficients
## Calculate R2 and adjusted R2 manually
TSS <- sd(mtcars$mpg)^2 * (n - 1)
# OR
TSS <- var(mtcars$mpg) * (n - 1)
(R2_R <- 1 - SSE_R/TSS)
(R2_R_adj <- 1 - (SSE_R/(n - k))/(TSS/(n - 1)))

# Interaction Terms
huxreg(
  lm(houseval ~ transit, data=californiatod),
  lm(houseval ~ transit * railtype, data=californiatod),
  lm(houseval ~ transit * region, data=californiatod),
  lm(houseval ~ transit * CA, data=californiatod))

# redefine the region variables with a new reference category (4 for SD)
catod2 <- californiatod %>% mutate(region = relevel(as.factor(region), ref = 4))
lm(houseval ~ region, data=catod2)  %>%  summary

# Partial F test:
catod3 <- californiatod %>% mutate(region = ifelse(region =="LA" | region == "SD", "LA_SD", region))
lm(houseval ~ region, data=catod3)  %>% summary
anova(lm(houseval ~ region, data=catod3), lm(houseval ~ region, data=californiatod))

# Hypothesis testing of linear combination of coefficients
car::lht(model_2_7_8, "x2 = x7")
# Partial F test:H0:β2+β2=0
car::lht(lm(hours ~ married*women, data=chores), "women + married:women = 0")

# linear combination of coefficients
# The point estimate is β2^+β3^ In this case, our linear combination involves the sum rather than the difference between two coefficients, and the formula for estimating the standard error of the sum of two coefficients is:
# $\sqrt{\hat{\sigma^2_{\hat{\beta_2}}} + \hat{\sigma^2_{\hat{\beta_3}}} + 2\hat{cov}_{\hat{\beta_2}\hat{\beta_3}}}$
fit1 <- lm(hours ~ married*women, data=chores)
beta2 <- coef(fit1)["women"]
beta3 <- coef(fit1)["married:women"]
betas_vcov <- vcov(fit1)
se <- sqrt(betas_vcov["women", "women"] + betas_vcov["married:women", "married:women"] + 2 * betas_vcov["women", "married:women"])
(t_stat <- (beta2 + beta3)/se)

## Degrees of Freedom
dof <- fit1$df.residual

## compare t_stat to critical t-value
(t_crit <- qt(0.025, df=dof, lower.tail = F))
## OR find the corresponding p-value
(p_val <- 2 * pt(t_stat, lower.tail = F, df=dof))

# Partial F test on the nonlinear term
anova(lm(houseval ~ density, data=californiatod),lm(houseval ~ density + I(density^2), data=californiatod))
#To be on the safe side, enclose your tranformation in an I() function. This is not necessary for log transformation.

library(olsrr)
# leverage (hat)
leverage <- ols_leverage(lm_sfr)
ols_rsdlev_plot(lm_sfr)
# Cook's distance
ols_cooksd_chart(lm_sfr)
# DFFITS
ols_dffits_plot(lm_sfr)
# DFBETAS
ols_dfbetas_panel(lm_sfr)
# Heteroskedasticity
ols_rvsp_plot(lm_sfr)
ols_rsd_qqplot(lm_sfr)
# hypothesis test of normality of residuals
ols_norm_test(lm_sfr
# Test of Heteroskedasticity with Breusch-Pagan Test
ols_bp_test(lm_sfr)
#Heteroskedasticity-Consistent Standard Errors
# standard variance-covariance matrix
vcov0 <- vcov(lm_sfr)
vcov(model_2_7_8)
# convert to correlation
vcov0
# Heteroskedasticity-Consistent variance covariance matrix
require(car)
vcov_hc3 <- hccm(lm_sfr, type="hc3")
# In presence of Heteroskedasticity, vcov_hc3 is larger than vcov0, to redo hypothesis tests
# with the Heteroskedasticity-Consistent variance covariance matrix
if (!require(lmtest)) install.packages("lmtest") & library(lmtest)
coeftest(lm_sfr, vcov_hc3)
# All possible subset
sfrmodel <- lm(TOTALVAL ~ BLDGSQFT + YEARBUILT + GIS_ACRES + dpioneer + dfwy + dpark + dmax + dbikehq, data = taxlot_sfr)
(sfrmodel_all_subset <- ols_all_subset(sfrmodel))
# Best Subset Regression
ols_best_subset(model_2_7_8)
# Multicollinary with VIF
ols_vif_tol(lm_sfr)
## Stepwise Forward Regression
# based on p-value
(sfrmodel_stepfwd.p <- ols_step_forward(sfrmodel))
# based on AIC
(sfrmodel_stepfwd.aic <- ols_stepaic_forward(sfrmodel))
## Stepwise Backward Regression
# based on p-value
(sfrmodel_stepbwd.p <- ols_step_backward(sfrmodel))
# based on AIC
(sfrmodel_stepbwd.aic <- ols_stepaic_backward(sfrmodel))
## Step AIC regression
# Build regression model from a set of candidate predictor variables by entering and removing predictors based on Akaike Information Criteria, in a stepwise manner until there is no variable left to enter or remove any more. The model should include all the candidate predictor variables.
(sfrmodel_stepboth.aic <- ols_stepaic_both(sfrmodel))

# Cross Validation: CV assesses how the results of a model will generalize to an independent data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.
library(modelr)
library(purrr)
(taxlot_sfr_kcv <- taxlot_sfr %>% 
  modelr::crossv_kfold() %>% 
  mutate(model=map(train, ~lm(TOTALVAL~BLDGSQFT+YEARBUILT+GIS_ACRES+dpioneer+dfwy, data=.x)),
         rmse=map2_dbl(model, test, modelr::rmse),
         rsquare=map2_dbl(model, test, modelr::rsquare)))
taxlot_sfr_kcv %>% 
  summarise_at(c("rmse", "rsquare"), funs(mean))

## DID omitted
## Discrete Outcome: Count/Poisson Regression
require(MASS)
require(huxtable)
fit_lm <- lm(carb ~ mpg + qsec, data=mtcars)
fit_glm <- glm(carb ~ mpg + qsec, data=mtcars, family="poisson")
huxreg(OLS=fit_lm, Poisson=fit_glm)

fit_lm <- lm(am ~ qsec + hp, data=mtcars)
fit_glm <- glm(am ~ qsec + hp, data=mtcars, family=binomial("logit"))
huxreg(OLS=fit_lm, logit=fit_glm)

# log Likelihood
logLik(fit_glm)
fit_glm0 <- update(fit_glm, .~1)
logLik(fit_glm0)
## 'log Lik.' -21.61487 (df=1)
# pseudo R2
1 - logLik(fit_glm)/logLik(fit_glm0)
## 'log Lik.' 0.381052 (df=3)
# Interpretation of coefficients
# odds ratio
(odds <- exp(coef(fit_glm)))
#prob
odds/(1 + odds)

huxtable::huxreg(model_2_7_8, statistics = NULL)

library(leaps) # Load the package #
model_wf_subset <- regsubsets(log(y) ~X2 + X3 +X4 + X5 + X6 + X7 + X8 + X9, data=table_wf, nbest=10 ) # nbest is the number of models from each size #
summary(model_wf_subset) # Hard to read output from this #

## plot adjusted R square for each model ##
plot(model_wf_subset, scale='adjr2')
## can use Cp, r2 or bic for scale ##
plot(model_wf_subset, scale='bic')
plot(model_wf_subset, scale='Cp')

shapiro.test(rstudent(model_wf_reduce_log)) #If p-value is bigger, then no problem of non-normality #
shapiro.test(rstudent(model_wf_reduce_log))

table_wf_resi <- table_wf %>% mutate(student_residual=rstudent(model_wf_reduce_log))
ggpairs(data=table_wf_resi[c(10,3,4,6,7,8,9,11)])

table_wf_resi <- table_wf %>% mutate(student_residual=rstudent(model_wf_reduce_log))
ggpairs(data=table_wf_resi[c(10,3,4,7,11)])


Anova(model_wf_final)
vif(model_wf_final)

confint(model_wf_final, level=0.05/1) # Bonferroni joint confidence interval #

plot(model_wf_final, pch=16, col="blue")
#Create Partial Regression plots #
avPlots(model_wf_final)

confint(model_wf_437, level=0.05/1) # Bonferroni joint confidence interval #

plot(model_wf_437, pch=16, col="blue")
#Create Partial Regression plots #
avPlots(model_wf_437)


deviation <- table_wf$y-mean(table_wf$y)

# Predit_Power=1-(PRESS.stat/SST)
1-((MPV::PRESS(model_wf_final))/(deviation%*%deviation)) # Compute SST by multiplying two vectors #
# prediction power of full
1-((MPV::PRESS(model_wf_reduce_log))/(var(table_wf$y)*(nrow(table_wf)-1)))
# prediction power of 437
1-((MPV::PRESS(model_wf_437))/(var(table_wf$y)*(nrow(table_wf)-1)))
# prediction power of backward
1-((MPV::PRESS(model_wf_final))/(var(table_wf$y)*(nrow(table_wf)-1)))
1-((ols_press(model_wf_437896_log))/(var(log(table_wf$y))*(nrow(table_wf)-1)))